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(54) Method and apparatus for recognizing inage pattern, method and apparatus for juding 

identity of image patterns, recording medium for recording the pattern recognizing method 
and recording medium for recording the pattern identity judging method 



(57) A plurality of teaching identification lace image 
patterns o1 teaching persons are obtained in a first pat- 
tern obtaining process using an image scanner, and a 
plurality of teaching video face image patterns o1 the 
teaching persons are obtained in a second pattern ob- 
taining process using a video camera, A feature extrac- 
tion matrix, which iTiiiiimizes an overlapping area be- 
tween a pattern distribution of the teaching Identification 
1ace image patterns and a perturbation distribution be- 
tween a group of teaching identification face image pat- 
terns and a group of teaching video face image patterns, 
is calculated. In cases where a feature extraction using 
the feature extraction matrix Is performed for referential 
lace image patterns of registered persons obtained in 
Ihc first pHllcrn oblHlning process, rclcrcntiHl ^c;^luro 
pattern vectors independent of any pattern obtaining 
process are obtained. When an input face image pattern 
of a specific person obtained in the second pattern ob- 
taining process is received, the feature extraction is per- 
formed lor the inputtace image pattern lo obtain an input 
feature pattern vector independent of any pattern ob- 
taining process. Therefore, the specific person can be 
recognized as a specific registered person by selecting 
a specific referential feature pattern vector most similar 
to the input feature pattern vector. 
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Description 

BACKGROUND OF THE INVENTION 
1. FIELD OF THE INVENTION: 



[0001] The present invention relates to a pattern recognizing method and a pattern recognizing apparatus in which 
an image ot a speech is recognized, a pattern identity judging method and a pattern identity judging apparatus in which 
it is judged according to the panem recognizing method v/hether or not an image or a speech is identical with another 
TO image or another speech, a recording medium lor recording a soltware program of the pattern recognition and a re- 
cording medium for recording h soflwHro program ol Lho prillcrn idcnlily judging method. 



2. DESCRIPTION OF THE RELATED ART: 



75 [0002] In a technical field of a pattern recognition such as a face image recognition or a speech recognition, second 
order statistics (or covariances) of model patterns are calculated from a set ol model patterns registered in a data base 
in advance, a pattern data space is made Irom the second order statistics, a distribution of an input pattern tn the 
pattern data space (that is, the portion occupied by an input pattem in the pattern data space) is assumed, andleatures 
of the input panem are extracted to recognize the input pattern. 



2.1. PREVIOUSLY PROPOSED ART 



[0003] For example, features ol the input pattern are extracted according to a well-known Karhunen-Loeve (KL) 
expansion method. This feature extraction is, for example, disclosed in a literature "M. Turk, A. Pentland, "Eigenfaces 

25 tor Recognition", Journal of Congnitive Neuroscience Volume 3. Number 1, 199V'. Though there are various other 
methods than the KL expansion method, the other methods are based on the KL expansion method. 
[0004] In the KL expansion method, each of two patterns Pa and Pb is approximated by a linear combination ol basis 
vectors (the number of vectors is N) Ei (i=1.2.-.N) to produce an approximated pattern, and the collation between the 
pallcrns Ph nnd Pb is performed by using approximnlcd pwlLcrns A rind B. The HpproximHLcd pHllcrns A and B Hrc 

30 formulated as follows. 

H 
N 



[0005] In the KL expansion method, a covariance matrix is obtained from W pieces of teaching pattern data an 
eigenvalue is calculated for each eigenvector of the covanaocainalrix, N eigenvectors corresponding to N higher 
eigenvalues (the number N is, for example, 100) are sSicted as N basis vectors Ei from all eigenvectors of the cov- 
^5 ariance matrix. 

[0006] In cases where a pattern data space is defined by the N basis vectors, there are two merits. 

(1) The W te aching pattern data projected on each plane defined by two basis vectors are separated from each 
other to a highest degree. Therefore, the W teaching pattern data can be easily distinguished Irom each other 
so (2) Noises included In the patterns Pa and Pb and changes occurring randomly In the patterns Pa and Pb can be 

removed. 

[0007] In the KL expansion method, it is supposed that an assuming precision for distribution parameters calculated 
from a pattem model set is sufficiently high. For example, in a face image recogniUon: in cases where a statistic property 
ss in a process for obtaining a pattern set agrees with that in a process for obtaining another pattem sat, many examina- 
tions indicate that a pattern recognition can be performed at a very high precision rate and the collation of the pattern 
sets can be correctly performed. 



2 



EP0 944 018 A2 



2.2. PROBLEMS TO BE SOLVED BY THE INVENTION: 

[OOOS] However, in cases ^.vhere features of a model pattern are extracted according to two types of image receiving 
processes, ttiere is a case that a first set of teaching pattern data obtained from the nnodel pattern according to the 

s first process greatly differs from a second set of teaching pattern data obtained from the same model pattern according 
to the second process, so that a statistic property for the first set of teaching pattern data greatly differs from that for 
the second set of teaching pattern data. For example, in cases where a lighting condition for photographing a first 
pattern differs from that tor photographing a second pattern, there is a case that a statistic propeiiy for a first set o1 
pattern data obtained from the first pattern differs from that for a second set of pattern data obtained from the second 

70 pattern . As a result, even though features of the first pattern agree with those of the second pattern, because an image 
rccognilion for Iho first and second scls of pHilcrn d^lrt is nol performed with sufficicnlly high precision, the collfiUon 
of the first and second sets of pattern data with each other is not correctly performed and the identity of the first pattern 
with the second pattern cannot be judged. 

[0009] The above problem is based on the supposition that tv/o pattern data sets compared v;ith each other are 
T5 derived from the common distribution (or the common statistic parameters). Therefore, in cases where two pattern 
data sets compared with each other are derived from different distributions (or different statistic parameters), the KL 
expansion method cannot be properly performed in the pattern recognition or the pattern collation. 

SUMMARY OF THE INVENTION 

20 

[001 0] A first object of the present invention is to provide, v/ith due consideration to the drav;backs of such conven- 
tional pattern recognizing method and apparatus, pattern recognizing method and apparatus in v/hich an input pattern 
identical with one of referential patterns is recognized with high precision even though a process lor obtaining the input 
pattern of an input sample differs from a process for obtaining the referential patterns from referential samples. 
^5 [001 1 ] A second object of the present invention is to provide pattern identity judging method and apparatus in which 
the identity of a first input pattern obtained according to a first process with a second input pattern obtained according 
to a second process is correctly judged regardless of a statistic property difference between the first and second input 
patterns occurred according to a difference between the first and second processes. 

[001 S] A Ihird object of Ihc present invcnlion is lo provide a recording medium in which n softwfirc program of the 
30 pattern recognizing method or a software program of the pattern identity judging method is recorded. 

[0013] The first object is achieved by the provision of a pattern recognizing method^ comprising the steps of: 

obtaining a set of first teaching patterns of a plurality of leaching samples according to a first pattern obtaining 
process; 

obtaining a set of second teaching patterns of the teaching samples according to a second pattern obtaining proc- 
ess differing from the first pattern obtaining process; 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second leaching 
patterns; 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of second 
40 teaching patterns; 

calculating a feature extraction matrix, which minimizes an overlapping area between the teaching pattern distri- 
bution and the teaching perturbation distribution, from the leaching pattern distribution and the teaching perturba- 
tion distribution; 

obtaining a set of referential patterns of a plurality ot referential samples according to the first pattern obtaining 
4S process; 

calculating a set of referential feature patterns of the referential samples from the set of referential patterns ac- 
cording lo ihc fcHlurc cxlraclion mnlrix, the scl of rcfcrcnliHl fcfjlurc pHlicrns being indopcndcnl of the first prtllorn 
obtaining process and the second pattern obtaining process; 

receiving an input pattern of an input sample according to the second pattern obtaining process; 
BO calculating an input feature pattern of the input sample from the input pattern according to the tealure extraction 

matrix; 

selecting a specific relerential feature pattern most similar to the input feature pattern from the set of referential 
feature patterns; £ind 

recognizing a specific referential sample corresponding lo the specific referential feature pattern as the input sam- 
ss pie. 

[0014] The first object is also achieved by the provision of a pattern recognizing apparatus, comprising: 
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tirst pmtern obtaining means for obtaining a set o1 first teaching patterns of a plurality of teaching samples according 
to a first pattern obtaining process; 

second pattern obtaining means for obtaining a set of second teaching patterns of the teaching samples according 
to a second pattern obtaining process differing from the first pattern obtaining process; 

5 feature extracting means for calculating a teaching pattern distribution fronn the set of first teaching patterns ob- 

tained by the first pattern obtaining means or the set of second teaching patterns obtained by the second pattern 
obtaining means, calculating a teaching distribution of a perturbation between the set of first teaching patterns and 
the set oi second teaching patterns, and calculating a feature extraction matrix, which minimizes an overlapping 
area between the teaching pattern distribution and the teaching perturbation distribution, from the teaching pattern 

10 distribution and the teaching perturbation distribution; 

rcfcrcnliHl fortlurc prtllorn cHlculHting mcnns for obfnining h set of rcfcrcnli^jl pHllcrns of h plurality of rcfcrcnllHl 
samples according to the first pallem obtaining process, and calculating a set of referential feature patterns of the 
referential samples from the set of referential patterns according to the feature extraction matrix calculated by the 
feature extracting means to make the set ol referential feature panems independent of the first pattern obtaining 

75 process and the second pattern obtaining process; and 

input panern recognizing means for receiving an input pattern of an input sample according to the second pattern 
obtaining process, calculating an input feature pattern ol the input sample from the'input pattern according to the 
feature extraction matrix calculated by the feature extracting means, selecting a specific referential feature pattern 
most similar to the input feature pattern from the set of referential feature patterns calculated by the referential 

^0 feature pattern calculating means, and recognizing a specific referential sample corresponding to the specific 

referential feature pattern as the input sample. 

[001S] In the above steps and configuration, in cases where a pattern obtaining process adopted to obtain a set of 
first panerns differs from a pattern obtaining process adopted to obtain a set of second patterns, a statistic property 
25 difference between the set of first pattems and the set of second patterns is generated. Therefore, even though one 
first pattern and one second pattern are obtained from the same sample, it is difficult to judge that the first pattern is 
identical with the second pattern. 

[0016] In the present invention, a feature extraction matrix, which minimizes an overlapping area between theteach- 
ing pHllcrn dislrlbulion and Ihc Ic^iching pcrLurbHlion dislribuLion, Is CHlcul^jlod. Therefore, in chscs where h feature 

30 extraction transformation using the feature extraction matrix is performed for the set of referential patterns to calculate 
the set of referential feature patterns, a referential pattern distribution and a referential perturbation distribution In the 
set of referential feature patterns has the same group ol distribution axes and become orthogonal to each other, so 
that perturbation components coming in the set of referential patterns can be removed in the set of referential feature 
patterns. This removal of the peilurbation components from the set of referential patterns denotes thai the set of ref- 

3$ erential feature patterns becomes independent of the first pattern obtaining process and the second pattern obtaining 
process. Also, the input feature pattern becomes independent of the first pattern obtaining process and the second 
pattern obtaining process. 

[0017] Accordingly, a specific referential sample corresponding to a specific referential feature pattern most similar 
to the input feature pattern can be correctly recognized as the input sample regardless of the tirst pattern obtaining 
40 process and the second pattern obtaining process. 

[0018] It is preferred that the step of calculating a teaching pattern distribution comprise the step of 

assuming a teaching pattern covariance matrix of a pattern sample space as the teaching pattern distribution, 
the step of calculating a teaching distribution of a perturbation comprise the steps of 
^5 calculating a teaching pattern perturbation between one first leaching pattern of one teaching sample and one 

second teaching pattern of the teaching sample for each teaching sample; and 

assuming a teaching pcrlurbatlon covariance matrix from Ihe leaching pallcrn pcrlurbalions as Ihc leaching per- 
turbation distribution, and 

the step of calculating a feature extraction matrix comprise the steps ol 
so calculating a both-diagonalizing matrix, which diagonallzes both the teaching pattern covariance matrix and the 

teaching perturbation covariance matrix, from the teaching pattern covariance matrix and the teaching perturbation 
covariance matrix; 

diagonalizing the teaching pattern covariance matrix according to the both-diagonalizing matrix to produce a di- 
agonal matrix of the teaching pattern covariance matrix; 
55 diagonalizing the teaching perturbation covariance matrix according to the both-diagonalizing matrix to produce a 

diagonal matrix of the teaching penurbation covariance matrix; 

calculating an amplitude re-transformation matrix, which again transforms a referential pattern covariance matnx 
indicated by the set of referential feature patterns to adjust amplitudes of diagonal elements of the referential 
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pattern covariance matrix alter the referential pattern covariance nnatrix is transfornned hy the both-diaqonalizing 
matrix to be diagonalized, fronn the diagonal matrices; and 

calculating the feature extraction matrix Irom the both<]iagonalizing matrix and the amplitude re-transformation 



matrix. 



[0019] Also, it is pi ef erred that the feature extracting means compiise 

pattern covariance assuming means for calculating a teaching pattern covariance matrix of a pattern sample space 
from the first teaching patterns or the second teaching patterns and assuming the teaching pattern covariance 

50 matrix as the teaching pattern distribution; 

pHllcm pcrturbHlicn crilculnting mcwns for cnlculHling h IcHching pHllcrn pcrlurbHtion bclwccn one Rrsl IcHching 
pattern of one teach ing sample and one second teaching pattern o( the teaching sample for each teaching sample; 
perturbation covariance assuming means for assuming a teaching perturbation covariance matrix Irom the teaching 
pattern perturbations calculated by the pattern perturbation calculating means as the teaching perturbation dislri- 

15 bution; 

both-diagonalizing matrix calculating means tor calculating a both-diagonaiizing matrix, which diagonalizes both 
the teaching pattern covariance matrix assumed by the pattern covariance assuming means and the teaching 
perturbation covariance matrix assumed by the perturbation covariance assuming means, Irom the teaching pat- 
tern covariance matrix and the teaching perturbation covariance matrix; 

^ diagonal matrix producing means fordiagonalizing the teaching pattern covariance matrix assumed by the pattern 

covariance assuming means according to the both-diagonalizing matrix calculated by the both <jiagonaliz ing matrix 
calculating means to produce a diagonal matrix o1 the teaching pattern covariance matrix, and diagonalizing the 
teaching perturbation covariance matrix assumed by the perturbation covariance assuming means according to 
the both-diagonalizing matrix to produce a diagonal matrix of the teaching perturbation covariance (natrix; 

2S amplitude re-transtormation matrix calculating means tor calculating an amplitude re-transformation matrix, which 

again transfoftns a referential pattern covariance matrix indicated by the set of referential feature patterns to be 
calculated by the referential feature pattern calculating means to adjust amplitudes of diagonal elements of the 
referential pattern covariance matrix after the referential pattern covariance matrix is transformed by the both- 
diHgonnli/ing matrix cHlcuk^lcd by Ihc bolh-dingonHli/ing matrix calcuhling mD;^ns to be di^^gonHli/cd, from the 

30 diagonal matrices; and 

calculating the leature extraction matrix from the both-diagonalizing matrix calculated by the both-diagonal izing 
matrix calculating means and the amplitude re-transformation matrix calculated by the amplitude re-transformation 
matrix calculating means. 

3S [0020] In the above steps and configuration, the teaching pattern covariance matrix derived from the set of first 
teaching patterns and the teaching perturbation covariance nnatrix derived Irom the set of first teaching patterns and 
the set of second teaching patterns have the same group of eigenvectors by diagonalizing the teaching pattern cov- 
ariance matrix and the teaching perturbation covariance matrix by using the both^iiagonal izing matrix. In addition, the 
diagonal elements of the teaching pattern covariance matrix and the diagonal elements of the teaching perturbation 

40 covariance matrix are adjusted by using the amplitude re-transformation matrix, so that not only the teaching pattern 
covariance matrix and the teaching perturt)ation covariance matrix have the same group of eigenvectors, but also the 
order of the eigenvectors arranged in the order of decreasing eigenvalues (or variance values) in the teaching pattern 
covariance matrix can be set to the reverse of the order of the eigenvectors arranged in the order of decreasing eigen- 
values (or variance values) in the teaching perturbation covariance matrix. 

45 [0021] Therefore, in cases where a feature extraction transfornnation using the feature extraction matrix is performed 
for the teaching pattern covariance matrix indicating a pattem distribution of the first teaching pattems and the teaching 
pcrturbHlion covHriancc mnlrix indicHling h pcrlurbHiion distribution bclwccn Ihc scl of firsl IcHching pnllcms nnd the 
set of second teaching pattems/a pattern sample space occupied by the pattem distribution has the same group of 
distribution axes (or the same group of basic vectors) as those of a pattern sample space occupied by the perturbation 

so distribution and the order of the spreading degrees ot the pattern distribution in directions of the axes Is the reverse 
of the order^ of the spreading degrees ot the perturbation distribution in directions of the axes. This reverse relationship 
in the spreading degrees between the pattern distributioi^ ar^d the perturbation distribution indicates a condition that 
the pattern distribution is orthogonal to the perturbation distribution, and an overlapping area between the pattern 
distribution and the perturbation distribution is minimized. The minimization of the overlapping area indicates that per- 

55 turbation components coming in the pattern sample space of the lirst teaching patterns are effectively removed. Be- 
cause the perturbation components denote a statistic property difference between the first teaching pattems and the 
second teaching pattems. a pattem recognition independent of a statistic property difference between the tirst obtaining 
process and the second obtaining pirDcess can be performed in cases where a feature extraction transformation using 
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the feature extraction matrix is perlormed lor the first leaching patterns. 

[0022] Also, it is prelerred that the step of calculating a leaching distribution of a perturbation comprise the step of 

calculating a teaching perturbation distribution between one first leaching panern of one teaching sample and one 
G second teaching pattern ot the teaching sample for each leaching sample^ 

the step of calculating a leature extraction matrix comprise the step of 

calculating afeature extraction matrix, which minimizes an overlapping area between a teaching pattern distribution 
of one teaching sample and the teaching peiturbattoii distribution ot the teaching sample, from the teaching pattern 
distribution and the teaching perturbation distribution o1 the teaching sample lor each leaching sample, 
TO the step ot obtaining a set of reterential patterns comprise the step of 

obLHining a sol ol rcfcrcnlial pnllcrns of Ihc teaching samples ^iccording lo Iho firsi pHLtorn oblHining process or 
the second pattern obtaining process. 

the step of calculating a set of referential leature patterns comprise the step ol 

calculating one referential feature pattern ot one teaching sample from one referential pattern of Ihe leaching 
T5 sample according to the lealure extraction matrix of the teaching sample for each teaching sample, 

the step of calculating an input feature pattern comprise the step ot 

calculating an input feature pattern of the Input sample from the input pattern according to the leature extraction 
matrix of one teaching sample for each teaching sample and 
the step ot selecting a specific referential feature pattern comprise the steps of 
20 estimating a similarity between one input feature pattern corresponding to one teaching sample and one referential 

feature pattern of the same teaching sample; and 

selecting a specific referential feature pattern ol a specific teaching sample most similar to the input feature pattern 
corresponding to the teaching sample fronn the set of reterential feature patterns. 

2$ [0023] Also, the first object is achieved by Ihe provision ol a pattern recognizing apparatus, comprising: 

first pattern obtaining means for obtaining a set ot first teaching patterns of a plurality ot registered samples ac- 
cording to a first pattern obtaining process; 

second pHllcm obtaining mcnns lor ablninlngH set of second icHching pHtlcrnsof Ihc registered sampics^iccording 
30 to a second pattern obtaining process dMfering from the first pattern obtaining process; 

feature extracting means for calculating a leaching pattern distribulbn from the first teaching patterns obtained by 

the first pattern obtaining means or the second leaching patterns obtained by the second pattern obtaining means. 

calculating a teaching distribution of a perturbation between one first teaching pattern ot one registered sample 

and one second teaching pattern of the registered sample for each registered sample, and calculating a feature 
^ extraction matrix, which minimizes an overlapping area between the teaching pattern distribution of one registered 

sample and the teaching perturbation distribution ol the registered sample, from the leaching pattern distnbution 

and the teaching perturbation distribution for each registered sample; 

referential feature pattern calculating means tor obtaining a set of referential patterns of the registered samples 
according to the first pattern obtaining process, and calculating a reterential feature pattern of one registered 

40 sample from one referential pattern of the registered sample according lo the feature extraction matrix ol the 

registered sample calculated by the feature extracting means for each registered sample to make each referential 
feature pattern independent of the first pattern obtaining process and the second pattern obtaining process; and 
Input pattern recognizing means tor receiving an input pattern of an Input sample according to the second pattern 
obtaining process, calculating an input feature pattern conesponding to one registered sample from the input 

45 pattern according to the feature extraction rrwtrix ot the registered sample calculated by the feature extracting 

means tor each registered sample, estimating a similarity between one reterential feature pattern of one registered 
SHmplc nnd Ihc inpul Icnlure pHllcm eorrcsponding lo the registered sample for cHch rcgislcrcd s;impto, selecting 
a specific referential feature paltern most similar lo the input feature paltem from the referential feature patterns 
calculated by the reterential feature pattern calculating means, and recognizing a specific registered sample cor- 

50 responding lo the specific referential feature paltern as the input sample. 

[0024] In the above steps and configuration, a leaching perturbatioii distribution between one first teaching pattern 
of one teaching sample and one second teaching panern ot the teaching sample is calculated for each teaching sample, 
a feature extraction matrix is calculated tor each teaching sample, and one reterential feature pattern corresponding 
55 to one teaching sample is calculated from one referential pattern ot the registered sample according to the first pattern 
obtainino process by using the feature extraction matrix for each teaching sample. Therefore: each referential feature 
pattern becomes independent ol the first pattern obtaining process and the second paltern obtaining process. 
[0025] When an input pattern of an input sample is received according lo the second pattern obtaining process, 
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because an input feature panern is calculated trom ihe input pattern according to the leature extraction matrix lor each 
registered sample, each input leature pattern becomes independent of the first pattern obtaining process and the 
second pattern obtaining process. 

[0026] Accordingly, even though the pattern obtaining process for obtaining the input pattern differs from that tor 
obtaining the referential patterns, in cases where a similarity between one referential feature pattem and one input 
feature pattern is estimated for each registered sample, a specific referential feature pattern most similar to the input 
feature pattem can be selected from the referential feature patterns, and a specific registered sample corresponding 
to the specific referential feature pattern can be recognized as the input sample. 

[0027] The second object is achieved by the provision of a pattern identity judging method, comprising the steps of: 

oblr^ining h set of first leeching pHLlcrns of h plurHlity of Icriching examples according to h lirsl pHllcrn oblaining 
process; 

obtaining a set of second teaching patterns of the teaching samples according to a second pattern obtaining proc- 
ess differing from the first pattern obtaining process; 
75 calculating a teaching pattern distribution from the set of first teaching patterns or the set ol second teaching 

patterns: 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of second 
teaching patterns; . . 

calculating a feature extraction matrix, which minimizes an overlapping area betv/een the teaching pattern distri- 
ct? bution and the teaching perturbation distribution, from the teaching pattern distribution and the leaching perturba- 
tion distribution; 

receiving a first input pattem of a first input sample according to the first pattem obtaining process; 

calculating a first input feature pattern of the first input sample from the first input pattern according to the feature 

extraction matrix, the first input feature pattern being independent of the first pattem obtaining process and the 

55 second pattem obtaining process; 

receiving a second input pattern of a second input sample according to the second pattern obtaining process; 
calculating a second input feature pattern of the second input sample from the second input pattem according to 
the feature extraction matrix, the second input feature pattern being independent of the first pattern obtaining 
process Ihc second pallcm oblaining process: 

30 collating the first input feature pattern with the second input feature pattem to estimate a similarity between the 

first input sample and the second input sample; and . . ^■ u 

judging that the first input sample is identical wrth the second input sample in cases where the similarity is high. 

[0028] The second object is also achieved by the provision of a pattem identity judging apparatus, comprising: 

3$ 

first pattem obtaining means for obtaining a set of first teaching patterns of a plurality of teaching samples according 
to a first pattern obtaining process; 

second pattern obtaining means for obtaining a set of second teaching patterns of the teaching samples according 
to a second pattern obtaining process differing from the first pattern obtaining process; 

40 leature extracting means for calculating a teaching pattem distribution from the set ol first teaching patterns ob- 

tained by the first pattem obtaining means or the set of second teaching patterns obtained by the second pattern 
obtaining mear^s. calculating a teaching distribution of a perturbation between the set of first teaching patterns and 
the set of second teaching patterns, and calculating a feature extraction matrix, which minimizes an overlapping 
area between Ihe leaching pattern distribution and the teaching perturbation distribution, from the leaching pattern 

4S distribution and the teaching perturbation distribution; 

leature pattern calculating means for receiving a first input pattern of a first input sample according to the first 
pHLlcrn oblHining process, receiving a second input pMtlcrn of a second input sannpio according lo the second 
pattern obtaining process, calculating a first input feature pattern of Ihe lirst input sample from the first input pattern 
according lo the feature extraction matrix calculated by the lealure extracting means to make the first input feature 

so pattern independent of the first pattem obtaining process and the second pattem obtaining process, and calculating 

a second input feature pattern ol the second input sample from the second input pattern according to the feature 
extraction matrix to make the second input feature pattem independent of the first pattern obtaining process and 
Ihe second pattern obtaining process; and 

identity judging means for collating the first input feature pattem calculated by the feature pattern calculating means 
55 With Ihe second input feature pattern calculated by the feature pattern calculating means to estimate a similarity 

between the first input sample and the second input sample, and judging that the first input sample is identical with 
the second input sample in cases where the similarity is high. 
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[0029] In the above steps and configuration, ihe feaiure extraction matrix is calculated in the same manner as in the 
pattern recognizing method. Therefore, the first input feature pattern and the second input feature pattern calculated 
by using the feature extraction matrix are independent of the first pattern obtaining process and the second pattern 
obtaining process. In this case, because the first input feature pattern derived from the first pattern obtaining process 
s can correctly collate with the second input feature pattern derived from the second patlern obtaining process, in cases 
where the first input sample is actually identical with the second input sample, the judgement that the first input sample 
is idenlical v/ith the second input sample can be reliably performed. 

[0030] The second object is also achieved by the provision of a pattern identity judging apparatus, comprising: 

to first pattern obtaining means for obtaining a set cf first teaching patterns of a plurality of teaching samples according 

lofi first pHtlcrn oblrtining process; 

second panem obtaining means for obtaining a group of second teaching patterns according to a second pattern 

obtaining process differing from the first pattern obtaining process for each teaching sample; 

feature extracting means for calculating a leaching pattern distribution from the set of first leaching patterns ob- 

75 tained by the first pattern obtaining means or the groups of second teaching patterns obtained by the second 

pattern obtaining means, calculating a teaching distribution of a peitut bation between one first teaching pattern of 
one teaching sample and the group of second teaching patterns of the teaching sample for each teaching sample, 
calculating an average teaching perturbation distribution from the teaching perturbation distributions, and calcu- 
lating a feature extraction matrix, which minimizes an overlapping area between the teaching pattem distribution 

20 and "the average teaching perturbation distribution, from the leaching pattern distribution and the average teaching 

perturbation distribution : . ,w ♦ 

feature pattem calculating means tor receiving a first input pattern of a first input sample according to the first 
pattern obtaining process, receiving a second input pattern of a second input sample according to the second 
pattem obtaining process, calculating a first input feature pattern of the first input sample f lom the first input pattem 

25 according to the feature extraction matrix calculated by the feature extracting means to make the first input feature 

pattern independent of the first pattern obtaining process and the second pattem obtaining process, and calculating 
a second input feature pattern of the second input sample from the second input pattem according to the feature 
extraction matrix to make the second input feature pattem independent of the first pattem obtaining process and 
Iho second pfiUcrn oblnining process; and 

3o identity judging means for collating the first input feature pattem calculated by the feature pattem calculating means 

with the second input feature patlern calculated by the feature pattern calculating means to estimate a similarity 
between the first input sample and the second input sample, and judging that the first input sample is identical with 
the second input sample in cases where the similarity is high. 

^ [0031] In the above configuration a group of second teaching panems are obtained according to a second pattem 
obtaining process for each teaching sample, a teaching perturbation distribution between one first teaching pattem of 
one teaching sample and the group of second teaching patterns of the teaching sample is calculated for each teaching 
sample an average teaching perturbation distribution is calculated from the teaching perturbation distributions, and a 
feature extraction matrix is calculated from the teaching pattem distribution and the average teaching perturbation 

-40 distribution. * *• * i 

[0032] Therefore, even though the pattern obtaining process for obtaining a first input pattern of a first input sample 
differs from that for obtaining a second input pattem of a second input sample, because a lirsi input feature pattem of 
the first input sample and a second input feature pattem of the second input sample are calculated according to the 
feature extraction matrix, the first input feature panem and the second input feature pattern become independent of . 

45 the first pattern obtaining process and the second pattern obtaining process. 

[0033] Accordingly, in cases where the first input feature pattern is collated with the second input feature pattern to 
• csIimHtc H simihrily between the VmbK input snmplc nnd the second input snmple. nnd the judgemoni whelhcr or not 
the first input sample is identical with the second input sample can be performed according to the similarity. 
[0034] The third object is achieved by the provision of a recording medium for recording a software program of a 

so pattern recognizing method executed in a computer, the pattern recognizing method, comprising the steps of: 

obtaining a set of first leaching patterns of a plurality of teaching samples acceding to a first pattem obtaining 
process; 

obtaining a set of second teaching patterns of the teaching samples according to a second pattern obtaining proc- 
55 ess differing from the first pattern obtaining process; 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second teaching 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of second 
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teaching patterns; 

calculating a feaiure extraction matrix^ ^vhich minimizes an overlapping area between the teaching pattern distri- 
bution and the teaching perturbation distribution, fronn the teaching pattern distribution and the teaching perturba- 
tion distribution; 

5 obtaining a set o1 relerential patterns of a plurality ot referential samples according to the first pattern obtaining 

process or the second pattern obtaining process; 

calculating a set of referential feature patterns of the referential sannples from the set of relerential patterns ac- 
cording to the leature extraction matrix, the set o1 referential feature patterns being independent of the first pattern 
obtaining process and the second pattern obtaining process; 
10 receiving an input pattern ot an input sample according to the first paltem obtaining process or ihe second pattern 

obl/jining process; 

calculating an input feature paltem of the input sample from the input pattern according to the feature extraction 
matrix; 

selecting a specific referential feature pattern most similar to the input feature panern from the set of referential 
^5 feature patterns; and 

recognizing a specific referential sample corresponding to the specific teferei^tial feature pattern as the input sam- 
ple. 

[003S] In the above recording medium, a software program of the pattern recognizing method can be recorded. 
20 Therefore, the software program of the pattern recognizing method can be executed in a computer. 

[0036] The third obiect is also achieved by the provision of a recording medium for recording a software program of 
a pattern identity judging method executed in a computer, the pattern identity Judging method, comprising the steps of: 

obtaining a set of lirst teaching patterns from a plurality of teaching samples according to a first pattern obtaining 
25 process; 

obtaining a set of second teaching patterns from the teaching samples according to a second pattern obtaining 
process differing from the first pattern obtaining process; 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second teaching 
pHllcms; 

30 calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of second 

teaching patterns; . 
calculating a feature extraction matrix, ivhich minimizes an overlapping area between the leaching pattern distri- 
bution and the teaching perturbation distribution, from the teaching pattern distribution and the leaching perturba- 
tion distribution; 

3S receiving a first input pattern of a first input sample according to the first pattern obtaining process; 

calculating a first input feature paltem of the first input sample from the first input pattern according to the feature 
extraction matrix, the first input feature pattern being independent of the first pattern obtaining process and the 
second pattern obtaining process; 

receiving a second input pattern of a second input sample according to the second pattern obtaining process; 
40 . calculating a second Input feature pattern of the second Input sample from the second input pattern according to 
the feature extraction matrix, the second input feature pattern being independent of the first pattern obtaining 
process and the second pattern obtaining process; 

collating the first input feature pattern with the second input feature pattern to estimate a similarity between the 
lirst input sample and the second input sample; and ■ 
45 iudging that the first input sample is identical with the second input sample in cases where the similarity is high. 

In the above recording medium, a software program of the pattern idemity judging method can be recorded. There- 
fore, the sollwnrc progrMm o( Lhc pallarn idcniily judging mclhod can be oxcculcd in a compulcr. 



SO 



BRIEF DESCRIPTION OF THE DRAWINGS 

[0037] The objects, features and advantages of the present invention will be apparent from Ihe following description 
taken in conjunction with the accompanying drawings, in which: 

Fig. 1 is a block diagram showing a hardware of a pattern recognizing apparatus additionally functioning as a 
panern identity judging apparatus according to the present invention: 

Fig. 2 is a block diagram showing a functional structure of a centra processing unit (CPU) arranged in the pattern 
recognizing apparatus according to a first embodiment of the present invention; 

Fig. 3 is a block diagram of a feature extraction matrix calculating means arranged in the CPU according to the 



9 



EP0 944 018 A2 

first embodiment; 

Fig. 4 shows an operation flow in an off-line processing initially performed in the pattern recognizing apparatus 
according to the first embodimenl; 

Fig. 5 schematically shows a condition that a pattern distribution is orthogonal to a perturbation distribution; 
5 Fig. 6 shows an operation flow in an on-line processing performed in the pattern recognizing apparatus according 

to the first embodiment; 

Fig. 7 is a block diagram showing a functional structure of the CPU arranged in the pattern identity judging apparatus 
according to a second embodiment of the present invention: 

Fig. 8 shov/san operation flow in an off-line processing initially performed in a feature extraction calculating means 
10 arranged in the CPU of the pattern identity judging apparatus according to the second embodiment; 

Fig. 9 shows «n opcrHlion Row in an on-line processing performed in h Icnluro pHllcrn oxlrHCling mcnns Hnd nn 
identity judging means arranged in the CPU of the pattern identity Judging apparatus according to the second 
embodiment; 

Fig. 10 is a block diagram showing a functional structure of the CPU 25 according to a third embodiment ol the 
75 present invention; 

Fig. 11 is a block diagram ol a feature extraction matrix calculating means arranged in the CPU according to the 
third embodimenl; 

Fig. 12 shows an operation flow in an off-line processing initially performed in the pattern recognizing apparatus 
according the third embodiment; 
20 Fig, 1 3 shows an operation flow in an on-line processing performed in the pattern recognizing apparatus according 

to the third embodiment; . . 

Fig. 14 is a block diagram showing a functional structure of the CPU 25 arranged in a pattern identity judging 
apparatus according to a forth embodiment of the present invention: 

Fig. 15 is a block diagram of a feature extraction matrix calculating means arranged in the CPU according to the 
25 fourth embodiment; 

Fig. 16 shows an operation flow in an off-line processing initially performed bi the pattern identity judging apparatus 

according the fourth embodiment: and 

Fig. 1 7 shows an operation flow in an on-line processing performed in the pattern recognizing apparatus according 
to the fourth embodiment. 



30 
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DETAILED DESCRIPTION OF THE EMBODIfv/IENTS 



[0038] Preferred embodiments of pattern recognizing method and apparatus, pattern identity judging method and 
apparatus and recorduig mediums according to the present invention are described with reference to the drawings. 

(First Embodiment) 

[0039] Fig. 1 is a block diagram showing a hardware ol a pattern recognizing apparatus additionally functioning as 
a pattern identity judging apparatus according to the present invention. 
40 [0040] As shown in Fig. 1 . a pattern recognizing apparatus 1 1 additionally lunctioning as a pattern identity judging 
apparatus comprises 

a video camera 12 for directly photographing a face of each of a plurality of teaching persons in a teaching image 
pattern obtaining process to obtain a plurality of teaching video lace images and directly photographing a face of 

4S a specific person in an image identity judging process to obtain an input lace innage; 

an image scanner 13 for scanning a face photograph of an identification card ol each teaching person in the 
IcHching image pnllcm oblnining process lo obtain h plurnlily of leaching idcnlificalion face images and scanning 
an idenliflcation face photograph of each of a plurality of registered persons in a referential image pattern obtaining 
process to obtain a plurality of referential identification lace images; 

so a computer system 14 for performing a pattern recognizing method according to the teaching video face images 

obtained by using the video camera 1 2 and the teaching identification face images and the referential identification 
face images obtained by using the image scanner 13 and performing a pattern identity judgu^g method to collate 
the input face image obtained by using the video camera 12 with each referential identification lace image and to 
identity the specific person; 

55 a keyboard 15 for inputting an instruction of an operator to the computer system 14; 

an auxiliary memory 16. embodied by an optical magnetic disc or the like, for registering a plurality ol referemial 

persons as members; , not^ 

a display 17 for displaying a face image determined in the computer system 14 as a recognized result ora coiiatea 
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result: and 

an output terminal 18 for outputting the face image deterrrtined in the computer system 14. 
[0041] The computer system 1 4. comprises 

3 

a video 1ace image memory 21 for temporarily storing the teaching video face images and the input face nnage 
transmitted from the video camera 1 2 through an interface (I/F) 22; 

an identification face image memory 23 tor temporarily storing the teaching idei^tification face images and the 
referential identification face images transmitted from the image scanner 13 through an interface (I/F) 24: 

to a central processing unit (CPU) 25 for converting each teaching video face image stored in the video face image 

memory 21 inlo a leeching video fficc imngc pHllcrn expressed by a onc-dimcnsionHl dalH string (or expressed 
by a pattern vector), converting each teaching identification lace image stored in the identification face image 
memory 23 into a teaching identitication face image pattern expressed by a one-dimensional data string (or ex- 
pressed by a pattern vector^, producing a feature esctraclion matrix f from the teaching video face image patterns 

75 and the teaching identification face image patterns, producing a referential identification face image pattern fronn 

each referential identification face image stored in the identification face image memory 23, producing a referential 
face image pattern from each referential identification face image pattern and the feature extraction matrix F, con- 
vening the input face image stored in the video face image menrvory 21 into an input lace image pattern expressed 
by a one-dimensional data siring (or expressed by a pattem vector), collating the input face image pattern vrith 

20 each referential face image pattem to judge whether or not the input face image pattern is identical with the ref- 

erential face image pattern, and judging that the specific person agrees with a specific registered person in cases 
where the input face image pattern is identical with the referential face image pattern corresponding to the specific 
registered person: 

a video lace image pattem memory 26 for storing the teaching video face image patterns and the rnput face image 
2S pattern obtained in the CPU 25; 

an identification face image pattern memory 27 for storing the teaching identification face image patterns and the 
referential face image patterns obtained in the CPU 25; 

a main memory 28= including a work area, for storing various software programs (including a software program ct 

H pHLlorn rccogni/ing mclhod or h soflwHrc progrnm of h pHllcm identity judging method) used in the CPU 25; 
30 a feature extraction matrix memory 29 for storing the feature extraction matrix F produced in the CPU 25; 

a referential face Image data base 30 for storing the referential face image data produced in the CPU 25: 

an interface (I/F) 31 connecting the keyboard 15 with the computer system 14; 

an interface (!/F) 32 connecting the auxiliary memory 16 with the computer system 14; 

an interlace (I/F) 33 connecting the display 17 with the computer system 14; and 
3S an interlace (I/F) 34 connecting the output terminal 18 with the computer system 14. 

[0042] Fig. 2 is a block diagram showing a functional staicture of the CPU 25 according to a first embodiment of the 
present invention. 

[0043] As shown in Fig. 2, the CPU 25 comprises 

40 

a feature extraction calculating means 41 for calculating the feature extraction matrix F from the teaching video 
face images and the teaching identification face images: 

a referential face image pattem determining means 42 for detemnining the referential face image patterns from 
the referential face images according to the feature extraction matrix F calculated in the feature extraction calcu- 
^5 laling means 41 : and 

an optimum match processing perlorming means 43 for pertorming an optimum match processing to judge whether 
or not Iho input fncc imngc pHllcrn of the specific person is idcniicHi with Ihc rcfcrcnliHl face imngc pHllcrn corre- 
sponding to one registered person. 

50 [0044] Fig. 3 is a block diagram of the feature extraction matrix calculating means 41 . 
[004S] As shown in Fig. 3, the feature extraction calculating means 41 comprises 

a pattern covariance assuming means 5i for assuming a pattem covariance matrix Cs of a pattem sample space 
from a set of the teaching video face image patterns and.'or a set of the teaching identification face image patterns; 
55 a pattern perturbation calculating means 52 for calculating 

a pattern perturbation Pi denoting a statistic property difference between one teaching video face image pattem 
and one teaching idenlitication face image pattem corresponding to one teaching person selected as a perturbation 
sample person for each perturbation sample person: 
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a perturbation covariance assuming means 53 tor assuming a covariance matrix Cp of the panern perturbations 
Pi calculated in the pattern perturbation calculating means 52: 

a both-diagona!izing matrix calculating means 54 tor calculating a bolh-diagonalizing matrix H. which diagonalizes 
both the pattern covariance matrix Cs and the perturbation covariance matrix Cp, from the pattern covariance 
s matrix Cs assumed in the pattern covariance assuming means 51 and the perturbation covariance matrix Cp 

assumed in the peituibalion covariance assuming means 53; 

a diagonal matrix producing means 55 tor diagonalizing the pattern covariance matrix Cs according to the both- 
diagonalizing matrix H calculated in the both -diagonalizing matrix calculating means 54 to produce a diagonal 
matrix Ds of the pattern covariance matrix Cs and diagonalizing the perturbation covariance matrix Cp according 

TO to the both-diagonalizing matrix H to produce a diagonal matrix Dp of the perturbation covariance matrix Cp; 

rin Hmpliludc rc-lrHnslormHlidn mnLrix cnlculnling means 56 for CHlculHling nn Hmpliludc rc-lrHnslormHlion mntrix 
L, which again transforms a pattern covariance matrix indicated by a plurality of panern vectors to adjust amplitudes 
o1 diagonal elements of the pattern covariance matrix after the pattern covariance matrix is transformed by the 
both-dlagonalizing matrix H to be diagonatized. from the diagonal matrices Ds and Dp produced in the diagonal 

75 matrix producing means 55; and 

a feature extraction rnalrix calculating means 57 lor calculating a feature extraction matrix F = LH from the both- 
diagonalizing matrix H calculated in the both-diagonalizing matrix calculating means 54 and the amplitude re- 
transformation matrix L calculated in the amplitude re-transfonmation matrix calculating means 56. 

20 [0046] In the above configuration, an operation performed in the pattern recognizing apparatus 11 is described ac- 
cording to a first embodiment. 

[0047] In this embodiment and following embodiments, the teaching identification face images ol the teaching per- 
sons are obtained according to a first image pattern obtaining process using the scanner 1 3. the teaching video face 
images of the teaching persons are obtained according to a second image pattern obtaining process using the video 
25 camera 1 2. and the referential face image pattern independent of a statistic property difference between the first image 
pattern obtaining process and the second image pattern obtaining process is produced from the teaching video tace 
images and the teaching identification face images. 

[0048] In the pattern recognizing apparatus 1 1 . the calculation of a feature extraction matrix F and the calculation ot 
. rclcrcnliHl Thcc image pHtlcrns using the icnturc cxlrdction matrix F arc inilially performed hs Hn off-line processing, 
30 and the judgement whether or not an input tace image of a specific person is identical with one referential face image 
of one registered person and the selection of the referential face image most similar to the input face image are per- 
tormed as an in-line processing. 

[0049] Fig. 4 shows an operation flow in an off-line processing initially performed in the pattern recognizing apparatus 

3S [0050] As shown in Fig. 4. a face ot a teaching person is directly photographed by the video camera 12 in a teaching 
image pattern obtaining process, so that an analog signal indicating a video face image ol the teaching person is 
obtained The analog signal is converted into a digital signal, and the digital signal is stored in the video face image 
memory 21 as a teaching video tace image (step SiOl). Also, aface photograph ol an identification card of the same 
teaching person is scanned by the image scanner 1 3 in Ihe teaching image pattern obtaining process, so that an analog 

40 signal indicating an identification face image of the teaching person is obtained. The analog signal is converted into a 
digital signal, and the digital signal is stored in the identification tace image memory 23 as a leaching identification 
tace image (step SI 01). 

[0051] The teaching video face image and the teaching identification face image are stored for each of a large number 
of teaching persons. The number of teaching persons is. for example. 15000 to obtain a sufficient number ol teaching 
45 lace images required tor the learning ol a feature extraction matrix F. Therefore, a plurality of teaching video face 
images ot 1 500 teaching persons are stored in the video face image memory 2 1 as a set of teaching video face images, 
rind H pturnlily of tcHching idcnlificnlion Ihcc images ol Ihc 1500 leaching persons arc stored in Ihc idcnliliCHlton face 
Image memory 23 as a set of teaching identification tace images. 

[0052] Thereafter, each teaching video face image is transfomned into a teaching video face image pattern expressed 
so by a one-dimensional data string in the feature extraction calculating means 41 of the CPU 25. and the teaching video 
face image patterns are stored in the video face image pattern memory 26 (step SI 02). Also, each teaching identification 
tace image is transformed into a teaching identification tace image pattern expressed by a one-dimensional data stung 
in the feature extraction calculating means 41, and the teaching identification tace image patterns are stored in the 
identification face image pattern memory 27 (step SI 02). 
55 [0053] Thereafter, a feature extraction matrix F is calculated in the feature extraction calculating means 41 in steps 
S 1 03 to SI 03. In detail, a panern covariance matrix Cs in a pattern sample space is calculated in the pattern covariance 
assuming moans 51 by using a set ol the teaching video (ace image patterns stored in the video lace image pattern 
memory 2S and/or a set of the teaching Identification tace image patterns stored in the identification face image pattern 
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memory 27 (step S103). The pattern covariance matrix Cs indicates statistics ot a pattern distribulion o1 a set o1 video 
tace image patterns and/or a set o1 identification lace image patterns in the pattern sample space. For example, in 
cases where a pattern co variance matrix C s in a pattem sample space is calculated by using a set of the teaching 
identificatton face image patterns, the pattern covariance matrix Cs is expressed according to a following equation. 

N 

cs=(l/N)2{bi-in)(bi-in)7 (2) 

i = l 



where bi denotes an identification pattern vector (one-dimensional data string) indicating the teaching identification 
face image pattern of the i-th teaching person, N denotes a number of samples corresponding to the teaching persons 
(N=15000). (bi - m)"^ denotes a Transposed matrix of (bi - m), and m denotes an average pattern vector expressed by 
75 a following equation 

U 

m - ( 1/N)2bi 

20 i = 1 ^ 

Also, in cases where a pattern covariance matrix Cs in a pattern sample space is calculated by using a set of the 
teaching video face image patterns and a set o1 the leaching Identification tace image patterns, the pattern covariaiK;6 
^5 matrix Cs is expressed acconding to a following equation. 

Cs = (i/2K)51{(ai - in)(ai - m)T + (bi - m){bi - m)T} (3) 
30 i=l 

where ai denotes a video pattern vector (one-dimensional data string) indicating the teaching video face image patterns 
of the i-th leaching pereon, and m denotes an average pattern vector expressed by a following equation. 

as 

n 

m = ( 1/2N)X* (ai + bi) 

40 

[O0S4] Thereafter, a pattern perturbation Pi denoting a statistic property difference between the teaching video tace 
image pattern and the teaching identification face image pattern in the i-th teaching person is calculated in the panem 
perturbation calculating means 52 acconding to a following equation. 

Pi = ai-bi (4) 

The pattern perturbation Pi occurs according to a statistic property difference between the first image pattern obtaining 
process using the image scanner 13 and the second image pattern obtaining process using the video camera 12. 
50 Thereafter, a perturbation covariance matrix Cp indicating a distribmion of a perturbation between a set of the teaching 
video tace image patterns and a set ot the teaching idemification tace image pattems is calculated in the perturbation 
covariance assuming means 53 by using the pattern perturbation Pi (step S104). 

Cp = (l/M)JPiPiT (5) 

i = 1 
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where M denotes a number of perturbation samples. The perturbation covariance matrix Cp is expressed by a covar- 
iance o1 the pattern perturbations Pi and indicates statistics o1 the change in a pattern distribution of a set o1 the teaching 
video face image patterns from a pattern distribution of a set of the leaching identification face image patterns. 
[00S5] Thereafter, an affine transformallon matrix H, which diagonali2es both ihe pattern covariance nnatrix Cs and 
the perturbation covariance matrix Cp, is calculated as a both-diagonal izing matrix in the both-diagonal izing matrix 
calculating means 54 by using the pattern covariance matrix Cs and the perturbation covariance matrix Cp according 
to a following equation (step Si 05). 

where <^ denotes an eigenvector matrix of the matrix Cs, A denotes an eigenvalue matrix of the matrix Cs, ^ denotes 
an eigenvector matrix of a matrix K defined in an equation 

K = (A ^|/)Cp(A <h ) 

and A-""'^ diag [o^'^''^. —1 is satisfied for the diagonal matrix a diag [a,, —J 

[00S6] Theieafter. a diagonal matrix Ds of the pattern covariance matrix Cs and a diagonal matrix Dp of the pertur- 
^ bation covariance matrix Cp are calculated in the diagonal matrix producing means 55 by diagonalizing the pattern 
covariance matrix Cs and the perturbation covariance matrix Cp according to the both-diagonalizing matrix H (step 
SI 06). 

2s Ds=HCsH^ 

Dp HCpH^ (7) 

where Ds Is expressed by Ds = diag [7^<^\ ?^(«>. —], and Dp is expressed by Dp = diag [>^(f>\ >.2<p). --]• 
[0057] Thereafter, an amplitude re-transfonnation matilx L, which again tiansforms a pattern covariance matrix in- 
dicated by a plurality o( pattern vectors to adjust amplitudes of diagonal elements of the pattern covariance matrix after 
the pattern covariance matrix is transformed by the both-diagonalizing matrix H to be diagonallzed. is calculated from 
the covariance diagonal matrices Ds and Dp in the amplitude re-transformation matrix calculating means 56 (step 
SI 07). 

L=diag[(X/ ) Af^2 '^^2 ^ • ^ 

where n denotes an arbitral real number. In this case, it is applicable that the amplitude re-transfomnalion matrix L be 
calculated according to a following equation (n=1). 

L = diag[t^, + a., ) . +^ ) '-"I 

[0058] Finally, a feature extraction matrix F Is calculated from the boih-dlagonallzing nr>airlx H calculated In the bolh- 
diagona lizing matrix calculating means 54 and the amplitude re-transformation matrix L in the feature extraction matnx 
calculating means 57 as a matrix product of L and H (step S108). and the matrix F is stored in the feature extraction 
matrix memory 29. The feature extraction matrix F is expressed according to a following equation. 



40 



45 



so 



55 



F = LH 



(10) 



[0059] Thereafter, referential face image patterns are calculated in the referential face image pattern determining 
means 42 in a referential image pattern obtaining process of the off-line processing. 

[0060] In detail, a face photograph of an identification card of each of a plurality of referential persons registered as 
members in the auxiliary storing unit 16 is scanned by the image scanner 1 3. and a referential identification face image 
is stored in the identification face imagejuerapry^ for each referential person. Thereafter, each referential identification 
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lace image is expressed by a one-dimensional data 3\mgJaj^^=2J,—M) as a referential identification tace image- 
pattern . and a referential feature pattern vector Ibi independent of a statistic property difference between the first image 
pattern obtaining process using the image scanner 1 3 and the second image pattern obtaining process using the video 
camera 12 is calculated by performing a fp-ature extraction Iranslormatio n using the feature^xtraction matrix J for the 
referential identification face image palterrTbTJtep SlOg). 



fbi Fbi C^) 

TO The referential feature pattern vecto(1bi^ stored in the referential face image data base 30 as a referential face image 
pfillcrn for cfach rcfcrcnliHl person. 

[0061] The reason that the referential feature pattern vector tbi Is Independent of a statistic property difference be- 
tween the first image pattern obtaining process using the image scanner 13 and the second image pattern obtaining 
process using the video camera 12 is described. 

75 [0062] Because the feat ure ex trac tion matrix F in the equation (11 ) is a product LH of the bolh-diagonalizing matrix 
H and the amplitude re-transforrrSion matrix L. the pattern covariance matrix Cs derived from the lefeiential identifi- 
cation face image panerns bjjor the referential video face image patterns ai) and the perturbation covariance matrix 
Cp derived from the referential identification face image patterns bi and the referential video face image patterns ai 
have Ihc snmc group of eigenvectors by diagonnli/ing the pHllcm covHrinncc mnlrix Csand the pcrlurb^lion cov/irwncG 

20 matrix Cp by using the both-diagonalizing matrix H of the f eature extraction matrix F. In addition, the diagonal elements 
of the pattern covariance matrix Cs and the diagonal elements of the perturbation covariance matrix Cp are adjusted 
by using the amplitude re-transformation matrix so that not only the pattern covariance matrix Cs and the perturbation 
covariance matrix Cp have the same group of eigenvectors, but also the order of the eigenvectors arranged in the 
order of decreasing eigenvalues (or var iance values) In the pattern covariance matrix Cs can be set to the reverse of 

2S the order of the eigenvectors arranged in the order of decreasing eigenvalues (or variance values) in the perturbation 
covariance matrix Cp. That is, in cases where the eigenvalues x,.Xo:-".x„ of the pattern covariance matrix Cs are 
Hrrnngcd in Ihc dccroHsing order x,>X2>X3>"->»<n. cigcnvHlucs yi.y2:"-:yn of !ho corresponding c.gcnvcclors in 
the perturbation covariance matrix Cp are arranged in the decreasing order yn>yn-,>"->y2>yi. Each eigenvalue in the 
covariance mnlrix indicates h varinncc (th;^l is. the sprc^^ding degree of a distribution) in h direction of Ihc corresponding 

30 eigenvector . . 

[0063] Therefore, in cases where a feature extraction transforniation using the feature exJracjionjiiaXrix F is per- 
formed for the pattern covariance matrix Cs indicating a pattern distribution of the face innage patterns and the penur- 
bation covariance matrix Cp indicating a perturbation distribution between a set of video face image patterns and a 
set of identification face image patterns, a pattern sample space occupied by the pattern distribution has the same 
3S group of distribution axes (or the same group of basic vectors) as those of a pattern sample space occupied by the 
perturbation distribution, and the order of the spreading degrees of the pattern distribution in directions of the axes is 
the reverse of the order of Ihc sprc^iding degrees of the pcrturbalion dislribulion in directions of Ihe hxcs. As shown 
in Fig 5. this reverse relationship in the spreading degrees behveen the pattern distribution and the perturbation dis- 
tribution indicates a condition that the pattern distribution is orthogonal to the perturbation distnbution. and an over- 
do lapping area between the pattern distribution and the perturbation distribution is minimized. The minimization of the 
overlapping area indicates that perturbation components coming in the pattern sample space of the face image pattems 
are effectively removed Because the perturbation components denote a statistic property difference between a group 
of image patterns obtained in a first process using the image scanner 13 and a group of image pattems obtained in a 
second process using the video camera 1 2. a pattern recognition independent ol a statistic property difference between 
45 Ihc first irriHgc pHllcrn oblnining process using Ihe imngo scanner 1 3 nnd the second imngc pattern obtaining process 
using the video camera 12 can be performed in cases where a feature extraction transformation using the feature 
extraction mHlrix F is performed lor Ihc imngc pHllerns. 

[0064] Thereafter in an in-line processing, the judgement whether or not an input face image of a specific person is 
identk:al with one referential face image of one referential person is pertomned for each registered person, and one or 

so more referential face images most similar to the input face image are selected in the optimum match processing per- 
forming means 43. ■ * n 
[0065] Fig 6 shows an operation How in an on-line processing pert ormed in the pattern recognizing apparatus n 
[0066] As shown in Fig. 6. a face of a specific person is directly photographed by using the video camera 12 to obtain 
an input f ace imag e, the input face image of the specific person is stored in the video lace image memory 21 (step 

55 S 1 1 1) Thcrc^Tll^hc input f^nn im^u yM^^rnnvortcd inio nn input face imnge pHttcjiLgJ^xprcssed by ^ onoKi.mcns,on;^t 
data siring in the CPU^25, and the in"^t face image panern 83 is stored in the vi3eb face image pattern memory 26 
(step 3112) Thereafter, an input feature pattern vecto(la2)ndependent of a statistic property difference between the 
first image pattern obtaining process using the Image scanner 13 and the second image pattern obtaining process 
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using ihe video camera 12 is calculaied by performing a leature extraction transiormation using the feaiure extraction 
matrix F for the input face image pattern (step S1 1 3). 



fa2 = Fa2 (12) 



ThcrcHflcr. Lhc judqcm cni whether or no l \hn input {nnlurc pallcm vcclor Ta o is idcnlicHl with one rcfcrcnLirjl fnco tmngc 
pattern is performed for'each referential face image pattern stored in the referential face image data base 30. and one 
or more referential lace image patterns mosi similar to ihe input feaiure pattern vector fggare selected as one or more 
recognized referential face image patlerns according to an optimum ma tch processing (step Sl1 4). Thereafter, one or 
more recognized face images indicated by the recognized referential face image patterns are displayed cn the display 
17 as one or more candidates for the specific person, ot the recognized referential face irnage paftems are output to 
the output te rminal 1 8 (step S 1 1 5). 

[0067] In this embodiment, a set of referential face image patterns is determined from a set of referential idenlrticatton 
^5 fHcc imngcs obtnincd by using lhc imngc scnnncr 13. However, bccHusc h fcHiurc oxlrnclion IrHnsformHlion using lhc 
feaiure extraction matrix F is performed for a set of face image patterns to obtain the set of referential face image 
pallcrns. even though ?i set of referential video fncc Imngcs -nrc oblrjincd by using the video c^mcrH 12. h sol of 
referential face image patlerns can be delermined from the set of referential video (ace images. 
[0068] Also, in this embodiment/the input lace image of the specific person is obtained by using the video camera 
^ 12 Hovf/ever, because a feature extraction transformation using the fe ature extraction matr ix F is performed for the 
input face image pattern, even though the input face image of the specific person is obtained by using the image 
scanner 13. the input feature pattern vector fa.^ independent of a statistic property difference between the first image 
pattern obtaining process and the second image pattern obtaining process can be obiained. 

[0069] Accordingly, because th e feature extraction matrix whi ch minim izes an o verlapping area between the pat- 

i?5 icrn dislribulionofHsc^^ 

assumed in advance by using a set of video face Images and a set of identification face images, a plurality of referential 
lace image patterns independent of a statistic property diflerence between the first image pattern obtaining process 
using the image scanner 13 and the second image pattern obtaining process using the video camera 12 can be ob- 
tained and one or more referential face image patterns most similar to a face image pattern corresponding to a specific 

^0 person can be selected regardless of whether the face image pattern is obtained according to the first image pattern 
obtaining process or the second image pattern obtaining process. 

[0070] In this embodiment, the video camera 12 and the image scanner 13 are used as image receiving means. 
Ho^vcvcr, the imnge receiving mcnns is nol limited lo the video camera 1 2 and the imngc scnnncr 1 3. nnd il is applicable 
that tvvo or more image receiving units be used as the image receiving means. 
^ [0071] Also, the set of rcfcrcnlial lace image patterns is oblHined in lhc off-lino processing. However, il is applicable 
that a set of referential face image patterns obtained in an external apparatus be stored in advance in the auxiliary 

storing unit 16. • i, < »• i 

[0072] Also, the set of referential face image patterns obtained in the off-line processing is stored tn the referential 
face image data base 30. However, rt is applicable that the set of referential face image patterns be stored in advance 
in the auxiliaiy storing unit 16. 

[0073] Also, in this embodiment, a face image pattern is recognized in the pattern recognizing apparatus 11 . However, 
it is applicable that an image pattern of a mobil car. pans or the like be recognized in the pattern recognizing apparatus 
11 . Also, il is applicable Ihat pallcmcd data oblaincd from audio dnla. character data or tho like be rccogni/cd in Lhc 
pattern recognizing apparatus li. 

(Second Embodiment) 

[0074] In a second embodiment, the judgement whether or not a specific person is identical with a photograph person 
corresponding to an identification card exhibited by the specific person is performed in a pattern identity judging ap- 

^ paratus according to a paUef^^tiWju^giaa.ni9m by coUaling a video face image pattern obtained by directly 
photographing a face of fhoipccifb pSrs^^n with a pholograph lace image pallcm oblaincd by scanning a face photo- 
graph of the identification card. A hardware of the pattern identity judging apparatus is the same as that of the pattern 
recognizing apparatus 11 shown in Fig. 1 . •» ■ ^ • ^ 

[0075] Fig. 7 is a block diagram showing a functional structure of the CPU 25 arranged in the pattern identity judging 

^5 apparatus ll according to a second embodiment of the present invention. 
[0076] As shown in Fig. 7, the CPU 25 comprises. 
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to 



15 



20 



25 



35 



the feature extraction calculating means 41 : 

ateature pattern extracting means 61 for performing a feature extraction transformation using the leatuf^^xtraction 
matrix F for a first input face image pattern-a^ol a first input face image, which is obtained by directly ^Qtographing 
al^ of a specific person with the video"3§mera ^2, to produce a first input feature pattern vecto^Ja^and per- 
forming aleatu re extraction transformation usingThe fe ature extrac tion matrix F for a second inpuH^ce image 
pattern to^l a second input face image, which is obtained by s cannin g a lace photograph of a photograph peison 
altachedfo an identification card with the image scanner 1 3, to produce a second input feature pattern v^ti^rtej^ip) 

^Identity judqing means 62 for collating the first input feature pattern vecto^^^ith the second input feature 
t^CiiS^t^ ^enmafo a cimiiaritu hptwcen the first incut feature Dattern vector fain and the second input 



pattern vectoffb^Eo estimate a s imilarity between the first input feature pattern vector fajn and the second input 
fcHlurc pHUc7nvcc& fbj^ nnd jud^ing^cording to Ihc similHrily whether or not Iho spccinc person corresponding 
to the first input feature pattern vector ts-^ is identical with the photograph person of the face photograph corre- ^ 
sponding to the second input feature pattern vector fbjn. 

[0077] In the above configuration, an operation performed in the pattern identity judging apparatus n is described. 
[0078] Fig. 8 shov/s an operation flow in an off-line processing initially peftcrmed in the f eature extrac tion calculating 
means 41 of the pattern identity judging apparatus 11. 

[0079] As shown in Fig. 8. the fP;^tureext raction matrix F is calculated in the steps S 101 to S 106 in the same manner 
as in the first embodiment. 

T0030] Thereafter when a specific person exhibits an ide ptiticatioo card, the judgement whether or not the specific 
Iperson direcUy photographed with the video camera 12 is identical with a photograph person of a lace photograph 

attached to the identification card is performed in an on-iine processing by collating a face image ol the specific person 
V^with a face image of the face photograph of the photograph person. 

[0031] Fig. 9 shows an operation flow in an on-line processing performed in the feature pattern extracting means 51 

and the identity judging means 62. 

[0032] As shown in Fig. 9. when a specific person exhibits an identification card, a face of the specific person is 
directly photographed with the video camera 12 to obtain .nfi^'^t innut face image, and the first input face innage of the 
specific person is stored in the video face image memory 21 (step S201 ). Also, a face photograph of the identification 
card is scann ed by the imngc scanner 1 3 Lo obl^^in h ^cond inoul (hco im^iqc. and Lhc second inpul f;.cc inn^gc of the 
face photograph is stored in the identification face image memory 23 (step S201). 

[0033] Thereafter, the first input face image is converted into a first input face image patterrSa^xpressed by a one- 
dimensional data string, and the first input face Image pattern Sj^ Is stored in the video face in^ p^tt^m memory 26 
(step S202) Also, the second input face image is converted into a second input face image pattertj^expressed by 
a one-dimensional data string, and the second input face image pattern b„ is stored in the iderUification face image 
pattern memory 27 (step S202). ^ — >^ 

[0034] Thereafter, a first input feature pattern vect^ fai„VideggQdent of a statistic property difference between the 
first image pattern obtaining process using the imagVsc"Snner 13 and the second image pattern obtaining process 
usinq the video camera 12 is calculated in the feature pattern extracting means 51 by performing a feature extraction 



transformation using the feature extractior 



for the first input lace image pattern aj^^fstep S203). 



la,. - Fa,. 



(13) 



45 



SO 



[0085] Also, a second input feature pattern wec\<(i^depeucieu\ of a statistic property difference between the first 
image pattern obtaining process using the image ^ner 13 and the second image pattern obtaining process using 
the video camera 1 2 is calculated in the feature pattern extracting means 61 by perfomning a feature extraction trans- 
formation using the feature e.xtraction mat rix F for the second input face image pattern bj^ (step S203). 



fb- Fbin 



(14) 



X0Q86] Thereafter, the first input feature pattern vect/mi^ collated with the second input feature pattern vector 
fbrin the identity iudging means 62 to estinnate a simiiaTttsTbetween the first input feature pattern vector fai„ and tne 



-sscbnd input leaturo pattern vector 1bi„. and it is judged according to th9 similarity whether or not the specrt>c person 
corresponding to the first input feature pattern vector lai„ is ideatical v/ith the photograph person ol the face Ptiotograpn 
corresponding to the second input feature paltarn vector fb^, (step S204). In cases where the similarity ,s sulticienlly 
high il is judged lh«l Ihc specific person is idcntic:il with the phologrnph person of the Ihcc photograph, .wnd h coiidi.an 



17 



EP 0 944 01 8 A2 



resull indicating that the specific person is identical with the photograph person o1 the face photo<?raph is displayed on 
the display 1 6 or is output to the output terminal 1 8 (step S205). !n contrast, in cases where the similarity is not sutficiently 
high, it is judged that the specific person is not identical with the photograph person of the face photograph, and a 
collation result indicating that the specific person is not identical wth the photograph person of the face photograph is 

5 displayed on the display 16 or is output to the output temninal 18 (step S205). 

[00S7] Accordingly, because the first input feature pattern vectofj^^iiidefDengscJ of a property difference 

between the Hrst image pattern obtaining process and the secondimage pattern obtainirfg^pf^ess and the second 
input featuie pattern vector fbj^ independent of the statistic property difference are obtained by peiforming a feature 
extraction transformation using the feature extraction matrix F for the Hrsl input face image pattem and the second 

10 input face image pattem bj^. and because the first input feature pattern vector fajn is collated with the second input 
Icalurc pHllom voclor fbj^. even though the processes for oblnining Ihc Iwo face images differ from CHch olhcr, Ihc 
Judgement whether or not the specific person of the photographed face bnage corresponding to the video camera 1 2 
is identical with the photograph person of the face photograph corresponding to the image scanner 1 3 can be correctly 
performed. That is, in cases where the specific person is actually identical with the photograph person of the lace 

75 photograph, a collation result indicating that the specific person is identical with the photograph person of the face 
photograph can be reliably obtained. 

[0038] In this embodiment, the video camera 12 and the image scanner 13 are used as image receiving means. 
However the image receiving means Is not limited to the video camera 1 2 and the image scanner 1 3. and it is applicable 
that two or more innaqe receiving units be used as the image receiving means. 
20 [0039] Also, the set of leaching face image patterns is obtained in the off-line processing. However, it is applicable 
that a set of teaching face image patterns obtained in an external apparatus be stored in advance in the auxiliary storing 

unit 16. ^ • u . t I 

[0090] Also, the set of teaching face image patterns obtained in the off-line processing is stored in the referental 
face image data base 30. However, it is applicable that the set of teaching face image patterns be stored in ad^/af^e 
2S in the auxiliary storing unit 16. 

[0091] Also, in this embodiment, two lace image patterns are collated with each other in the pattern identity judging 
apparatus 1 1 However, it is applicable that image patterns of a mobil car parts or the like be collated with each other 
in the pattem identity judging apparatus 11 . Also, it is applicable that patterned data obtained from audio data= character 
dal« or the like be colinlcd wiih c^ich other in Ihe pHllcrn idcnlily judging ^^pp^rnlus 11 . 

30 

(Third Embodiment) 

[0092] In this embodiment, a feature extraction matrix is produced 1or,fia£h registered person, a referential face 
irp^ig^p^^^^^,. f^.».hi^hofoc.t..ro«virarfinniifiin9thecofresDondinQfeatureextractionrnatr^ is produced 

3S tor each registered person, and a specific person, of which an input face image pattern is most similar to a specific 
referential face image pattern of a specific registered person selected from the registered persons, is recognized as 
the specific registered person by using the pattern recognizing apparatus shown in Fig. 1. 

[0093] Fig. 10 is a block diagram showing a functional structure of the CPU 25 according to a third embodiment o1 
the present invention. 
40 [0094] As shown in Fig. 10. the CPU 25 comprises 

a feature extraction calculating ineans 71 for calculating a feature extraction matrix F i from a plujality of teaching 
video face images of a plurality of registered persons and a plurality of teaching identiftcation face images of the 
registered persons; 

4S a referential face image pattem determining means 72 for determining a referential face image pattern from a 

referential face image of one registered person according to the feature exlracUon matrix Fi of the registered person 
CHlcubiicd in Lhc fcHlurc cxlrnclion calculnling mcnns 71 (or each of the rogislcrcd persons; nnd 
an optimum match processing performing means 73 for performing an optimum match processing to judge whether 
or not an input face image pattem of a specific person is identical with the referential face image pattem of each 

so registered person and recognizing the specific person as a specific registered person selected from the registered 

persons on condition that the input face image pattern of the specific person is most similar to a specific referential 
face image pattern of the specific registered person. 

[0095] Fig. 1 1 is a block diagram of the feature extraction matrix calculating means 71 . 
55 [0096] As shown in Fig. 1 1 , the feature extraction calculating means 71 comprises 

a pattern covariance assuming means B1 for assuming a pattern covariance matrix Cs of a pattern sample space 
from a set ot the teaching video face image patterns and^or a set of the teaching identification face Image patterns: 



18 



EP 0 944 018 A2 

a pattern periurbalion calculating means 82 tor calculating a pattern perturbation Pji denoting a statistic property 
dilference between one teaching video (ace image paltern aji and one teaching identification face image pattern 
bi corresponding 1o one registered person lor each registered person; 

a perturbation covariance assuming means 63 for assuming a covariance matrix Cp(i) of the pattern perturbations 

5 Pji (j=i to L) calculated in the pattern perturbation calculating means 82 for each registered person; 

a both-diagonalizing matrix calculating means 84 for calculating a both-diagonalizing matrix Hi. which diagonalizes 
both the pattern covariance matrix Cs and the perturbation covariance matrix Cp(i), from the pattern covariance 
matrix Cs assumed in the pattern covariance assuming means 51 and the perturbation covariance matrix Cp(i) 
assumed in the perturbation covariance assuming means 82; 

70 a diagonal matrix producing means 35 lor diagonalizing the pattern covariance matrix Cs according to the bothdi- 

agonHli/ing nvjlrix Hi calculated in Ihc boih-diagon^ili/ing mrslrix cnlcul/iling means 34 to produce h diagonnl mnlrix 
Ds(i) of the panern covariance matrix Cs for each registered person and diagonalizing the perturbation covariance 
matrix Cp(i) according to the both-diagonalizing matrix Hi to produce a diagonal matrix Dp(i) o1 the perturbation 
covariance matrix Cp(i) tor each registered person; 

15 an amplitude re-transtormation matrix calculating means 96 for calculating an amplitude re-transtormation matrix 

Li, whictiagain transforms a pattern covariance matrixindicated by a plurality of pattern vectorsto adjust amplitudes 
ot*diagonai elements of the pattern covariance nnalrix after the pattern covariance matrix is transformed by the 
both-diagonalizing maUix Hi to be diagonalized from thediagonal matrices Ds(i) and Dp(i) produced in the diagonal 
matrix producing means B5 tor each registered person; and 

20 a feature extraction matrix calculating means 87 for calculating a feature extraction matrix Fi = LiHi from the both- 

diagonalizing matrix Hi calculated in the both-diagonaliz ing matrix calculating means 84 and the amplitude re- 
transforrTi ation matrix Li calculated in the amplitude re-transtormation matrix calculating means 36 for each reg- 
istered person. 

2S [0097] In the above configuration, an operation performed in the pattern recognizing apparatus 11 is described ac- 
cording to a third embodiment. 

[O098] In the pattern recopnizing apparatus 11. the calculation of feature extraction matrices Fi and the calculation 
of referential face image patterns using the feature extraction matrices Fi are initially performed as an off-line process- 
ing, Hnd the judgcmcnl whether or nol an Input fnco imngo of a specific person is idcnlic^^l with one rcfcrcnlifjl face 
30 Image of one registered person and the recognition of the specific person as a specific registered person are performed 

as an in-line processing. . . 

[0099] Fig. 12 shows an operation flow In an off-line processing initially performed in the pattern recognizing appa- 
ratus 1 1 according to the third embodiment. 

[0100] As shown in Fig. 12. a face of a registered person is directly photographed by the video camera 12 in a 
teaching image pattern obtaining process, so that an analog signal indicating a moving picture composed of a series 
of still pictures is obtained. A face innage ot the registered person is depicted in each still picture as a video face image. 
The analog signal is converted into a digital signal, and the digital signal is stored in the video face image memory 21 
as a group of "teaching video face images (step S301). Also, a face photograph of an identification card of the same 
registered per^n is scanned by the image scanner 1 3 in the teaching image pattern obtaining process, and a teaching 
40 identification face Image of the registered person is stored in the identification face image memory 23 in the same 
nnanner as in the first embodiment (step S301 ). 

[0101] The teaching video face image and the teaching identification face image are stored tor each of registered 
persons. The number of registered persons is, tor example. 15000 to obtain a sufficient number of teaching face images 
required for the learning of feature extraction matrices Fi. Therefore, a plurality of groups of teaching video face images 
4S of 1 500 registered persons are stored in the video face image memory 21 as a set of teaching video face images, and 
a plurality of teaching identification face images of the 1500 registered persons are stored in the Identification face 
imngc memory 23 hs h set of IcHching idcnIificMlion (hcc images. 

[0102] Thereafter, each teaching video face image is transformed into a teaching video face image pattern aji ex- 
pressed by a one-dimensional data string in the feature extraction calculating means 7 1 of the CPU 25, and the teaching 

so video face image patterns aji are stored in the video face image pattern memory 26 (step S302). A subscript -j" of aji 
corresponds to each still picture of one registered person ij i to L). and a subscript "in of aji corresponds to each 
registered person (i=1 to N). Also, each leaching identification face image is transfoimed into a teaching identification 
face image pattern bi in the same manner as in the first embodiment in the tealure extraction calculating means 71 , 
and the teaching identification face image patterns bi are stored in the identiftcation face image pattern memory 27 

55 (step S302). 

[0103] Thereafter, a feature extraction matrix Fi is calculated tor each registered person in the feature extraction 
calculating means 71 in steps S303 to S308. In detail, a pattern covariance matrix Cs defined in a paltern sample 
space is calculated in the pattern covariance assuming means 91 by using a set of the teaching video face image 
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patterns aji stored in the video face image pattern memory 26 and/or a set o1 ihe teaching identificalion face image 
patterns bi stored in the identilication face image pattern memory 27 (step S303). For example, in cases where a 
pattern covariance matrix Cs is calculated by using a set o1 the leaching identitication lace image patterns bi. the 
pattern covariance matrix Cs is expressed in the equatbn (2) in the same manner as in the first embodiment. 
5 [0104] Also, in cases where a paiiem covariance matrix Cs is calculated by using a set of the teaching video face 
image patterns, the pattern covariance matrix Cs is expressed according to a following equation. 



cs = (l/M)2{ai - n)(ai - m)T (L5) 

i = l 

where ai denotes an average pattern vector c1 each registered person expressed by a following equation 

1$ 

ai = {1/L)2aji 

20 

and m denotes an average pattern vector expressed by a following equation 



2$ 



m 



11 



30 Also. In cases where a pattern covariance nnatrix Cs in a pattern sample space is calculated by using a set of the 
teaching video face image patterns and a set o1 the teaching identification face image patterns, the partem covariance 
matrix Cs is expressed in the equation (3) in the same manner as in the first embodinnent. 

[0105] Thereatter, a pattern perturbation Pji denoting a statistic property difference betv-reen each teaching video 
face image pattern aji of the i-th registered person and one teaching identification face image pattern bi o1 the i-th 
35 registered person is calculated in the pattern perturbation calculating means 82 according to a following equation. 



Pji = aji - bi 



(16) 



Thereafter, a perturbation covariance matrix Cp(i) indicating a distribution of a perturisalion between one group of 
teaching video face image patterns a ji and one teaching identitication lace image pattern bi is calculated for each 
registered person in the perlurbation covariance assuming meansas by using the pattern perturbations Pji (step S304). 



45 



Cp(i) = (l/L)2PjiPjiT 
j = l 



— (17) 



50 Where L denotes the number of still pictures existing in one moving picture. Each perturbation covariance matrix Cp 
(I) is expressed by a covariance of the pattern perturbations Pji and indicates a statistic difference in a pattern distribution 
of one registered person bet\veen a face image obtained from the video camera 12 a.KJ a face image obtained from 
the image scanner 10. ■ « ^ 

[0106] Thereafter, an affine transformation matrix Hi. which diagonalizes both the pattern covariance matrix Cs ana 

55 the pcrlurbrilion covHririncc matrix Cpfi). is c^ilculnlcd hs h bolh-di^igonHli/ing mHlrix for c;ich registered person tn Lhc 
both-diagonalizing matrix calculating means 64 by using the pattern covariance matrix Cs and the perturbation covar- 
iance matrix Cp(i) according to a follov/tng equation (step S305). 
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Where 4* denotes an eigenvector matrix of a matrix Ki defined in an equation 

-1"? T -1'2 T T 

Ki = A )Cp(iKA * $ i . 

[0107] Theiealter, a diagonal matrix Ds(i} of the pattern covariance matrix Cs and a diagonal matrix Dp(i) of the 
perturbation covariance matrix Cp(i) are calculated for each registered person in the diagonal matrix producing means 
85 by diagonalizing the pattern covariance matrix Cs and the pe'rturbation covariance matrix Cp(i) according to the 
both-dia penalizing matrix Hi (step S305). 

^5 Ds(t) = HiCsHi^ 

Dp(i) = HiCpCilHi"^ (TS) 

Where Ds (i) is expressed by 

Ds(i) = diag [X.i*^*/ Kz^^* ' 1' 

Hnd Dp(i) is expressed by 

Dp(i) = diag [X^ii?*, /-i2(P'/ ^^3-^^' 

30 

[0108] Therealter. an amplitude re-transformation matrix Li. which again transforms a pattern covariance matrix in- 
dicated by a plurality ol pattern vectors to adjust amplitudes o1 diagonal elements ol the pattern covariance matrix after 
the pattern covariance matrix is transformed by the both-diagonalizing matrix Hi to be diagonalized. is calculated from 
the covariance diagonal matrices Ds(i) and Dp(i) for each registered person in the amplitude re-lran storm ation matrix 
^ calculHting moans £6 (slop S307). 

Li = diag [(A,;t^i-"- + (>n3(^»" + ^.^j'^'M"^'^". ---1 

40 ' ^ (20) 

whore n denotes an Hrbilr^l real number. In this case, it is applicable lhal the amplitude re-lransformHlion matrix Li be 
calculated according to a following equation (n=1). 

45 

Li ~ diag [(^^(s) + >,^^(p) ) -1/2 , (X^2(s) + ] 

— (21) 

60 

[0109] Finally a feature extraction matrix Fi is calculated from the both-diagonalizing matrix Hi calculated in the both- 
diagonalizing matrix calculating means 84 and the amplitude ro-lransformalion matrix Li for each rcgislorod person in 
the leature extraction matrix calculating means 87 as a matrix product of Li and HI (step S308). and the matrices Fl 
are stored in the feature extraction matrix memory 29. Each feature extraction matrix Fi is expressed according to a 
lollowing equation. 
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Fi = LiHi (22) 

[0110] Thereaner, reterential tace image patterns are calculated in the referential face image pattern determining 

5 means 72 in a reterential image partem obtaining process of the off-line processing. 

[0111] In detail, a face photograph ot an identification card of each of the registered persons is scanned by the image 
scanner 10^ and a face image o1 each registered person is stored as a referential identification face image in the 
identification face image memory 23 lor each registered person. Thereafter, each referential identification face image 
is expressed by h one-dim cnsionn I dnlH siring B^i hs h rcfcrcnlifil idcnlificnlion (hco imngc pfitlcrn, rind h rcfcrcnliHl 

10 feature pattern vector 1B,i independent of a statistic property difference between the first image pattern obtaining 
process using Lhc im^gc scanner 13 Hnd Ihc second imago pnllcrn oblHining process using Ihc video camera 12 is 
calculated for each registered person by performing a feature emraction transfcnmaticn using the feature extraction 
matrix Fi for the referential identification face image pattern B-ji (step S309). 

fB,i FiB^i (23) 

Where the feature extraction matrix Fi and the referential identification face image pattern B^i correspond to the same 
registered person. 

^ [011 2] The referential feature pattern vector tBi i is stored in the referential face image data base 30 as a reterential 
face image pattern for each registered person. 

[0113] The reason that the referential feature pattern vector fBii is independent of a statistic properly difference 
between the first image pattern obtaining process using the image scanner 1 3 and the second image pattern obtaining 
process using the video camera 12 is described. 

^5 [0114] BccHusc lhc fcHlurc cxlrHclion nrwLrix Fi in the oqunlion (11) is h product LiHi of lhc bolh-diagonHli/tng mnlrix 
Hi and the amplitude re-transtormation matrix Li. an i-th registered person component of the pattern covariancs matrix 
Cs derived from the referential identification face image patterns bi (or the referential video face image patterns ai) 
and the perturbation covariartce matrix Cp(i) deri-v/ed from the referential identification face image pattern bi and the 
referential video face image pattern ai have the same group o1 eigenvectors by diagonaiizing the i-th registered person 

^ component of the pattern covariance matrix Cs and the perturbation covariance matrix Cp(i) by usingthe both-diago- 
nalizing matrix Hi. In addition, the diagonal elements of the i-th registered person component of the pattern covariance 
matrix Cs and the diagonal elements of the perturbation covariance nnairix Cp(i) are adjusted by using the amplitude 
rc-lransrormalion malrix Li. so Ihwl nol only Lhc. i-lh rcgisLcrcd person componcnl of lhc pnllcrn covarinncc matrix Cs 
and ih e perturbation covariance matrix Cp(i) have the same grou p of eigenvectors, but also th e order of th e eigenvectors 

^5 arranged in the order of decreasing eigenvalues (or variance values) in the i-lh registered person component of the 
pattern covariance matrix Cs can be set to the reverse of the order of the eigenvectors arranged in the order of de- 
creasing eigenvalues (or variance values) in the perturbation covariance matrix Cp(i). That is. in cases where the 
eigenvalues x,,X2,"-^n o\ the i-lh registered person component ot the pattern covariance matrix Cs are arranged in 
the decreasing order Xi>X2>X3>—>x„, the eigenvalues yi,y2.—.yn of the corresponding eigenvectors in the perturbation 
covariance matrix Cp(i) are arranged in the decreasing order y,^yn.i>— >y2>yi- ^^^^ eigenvalue in the covariance 
matrix indicates a variance (that is, the spreading degree ot a distribution) in a direction of the corresponding eigen- 
vector. 

[0115] Thercforo. in chscs where n fcHluro cxIrHClion IrnnsformHlion using Iho fcrilurc oxlrHcLion mnlrix Fi is per- 
formed for the i-th registered person component ot the pattern covariance matrix Cs indicating a pattern distribution ot 

-'^ the face image patterns and the perturbaiion covariance matrix Cp(i) indicating a perturbation distribution l:>etween the 
video face image pattern ot the i-th registered person and the identification face image pattern of the i-th registered 
person, a pattern sample space occupied by the pattern distribution has the same group of distribution axes (or the 
same group ot basic vectors) as those of a pattern sample space occupied by the perturbation distribution, and the 
order of the spreading degrees of the pattern distribution in directions of the axes is the reverse of the order ot the 

^ spreading degrees ot the perturbation distribution in directions of the axes. This reverse relationship in the spreading 
degrees bclwccn lhc pHllcrn dislribulion of lhc i-lh rcgislcrcd person Hnd Lhc pcrlurbalion dislribulion ol Lhc i-lh reg- 
istered person indicates a condition that the pattern distribution is orthogonal to the perturbation distribution, and an 
overlapping area between the pattern distribution of the i-th registered person and the perturbation distribution ot the 
i-lh registered person is minimised. The minimization ot the overlapping area indicates that perturbation components 

^5 coming in the pattern sample space of the face image pattern ol the i-th registered person are effectively removed. 
Because the perturbation components denote a statistic propeny difference between the image pattern of the i-th 
registered person obtained in a first process using the image scanner 1 3 and the image pattern of the i-lh registered 
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person obtained in a second process using the video camera 1 Z. a pattern recognition independent of a statistic property 
difference between the first image pattern obtaining process using the image scanner 1 3 and the second image pattern 
obtaining process using the video camera 12 can be perlcrmed in cases where a feature extraction transformation 
using the feature extraction matrix Fi is pertormed for each image pattern. Thereafter, in an in-line processing, the 
5 judgement whether or not an input face image of a specific person is identical with one referential face image of one 
registered person is performed \ox each registered person, and one or more referential face images most similar to 
the input face image are selected in the optimum match processing performing means 73. 

[0116] Fig. 13 shows an operation flow in an on-line processing perlormed in the pattern recognizing apparatus 11 
according to the third embodiment. 

70 [0117] As shown in Fig. 13, a lace of a specific person is directly photographed by using the video canriera 12 to 
oblHin Hn inpul moving picture composed of n scries of input slill pictures of Ihc specific person. A frtcc imago of the 
specific person is depicted in each still picture as an input face image. A group of inpul face images of the specific 
person argjiloFodini he videp^ce ima q9-m.Qmory-21 (step S311 ). Thereafter, each input faca image is converted into 
an InputTacrTrn^ge-patt e ri(rAJ ^p ressed by a one-dimensional data siring in the CPU 25, and the group c 1 ipputface 

75 image patternsA^jyr^ sior^S^he video face image pattern memory 26 (step S312). Thereafter a feature extraction 
transformati^njiSTfig the feature extraction r ^\x\r^ o\j^j&^\^\^\^^ person is peilormed for the inpul face image 
paltern''S3^ calculate an inpuUeature pattern veSo^Wji^Vresponding to the registered person (step S313) from 
the inpuU^e image patter ri^^2i;^^ly' 

^ fAgji^FiA^i (24) 

Where the feature ex traction matrix F i corresponds to the i-th registered person, and m^jnput lace image pattern Agj 
corresponds to one still picture of the specific person. The input feature patiorn vector^^ji)s c alculated for each input 

i?5 f„cG irrwgc pHtlcrr^^gi)^^^' 'S- P'Clurc of Ihc specific person). Also. Lhc inpul fcHlurc pHllcrn vector fAoji 

is calculated tor eacJrfeature e)ftraclion matrix Fi (that is, for each registered person). The inpuUeature pattern vectors 
1 Agji are in dependent ot a statistic property difference -bet ween the first imag e pattern obtaining process usinqjhe 
ima^ge_scanneOJiand^ second image^attem obtaining process using the video camera 12. 
[Oil 8] Thereafter, an average input feature pattern vector corresponding to each registered person is calculated 

30 according to a following equation. 
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L 

fAji = (l/LlS^A^ji — - (25) 



[Oil 9] Thereafter, a similarity between the average inpul feature pattern vectt^f A2i_cj>rresponding to one registered 
person and one referenfiafface image patter^Ti^orresponding to the same registered person is estimated according 
to an optimum match processing lor each relemrtfial face image pattern stored in the referential face image data base 
30 (that iS: for each registered person) to judge vvhether or not the average input feature pattern vect or fA:>i co rre- 
spohding to one registered person is identical with one referential face image pattern fBii corresponding to the same 
rcgislcrcd person, and ono or more rcfcrcnliril face im^gc p^^llcrns rcspccLivcly similar lo Lhc averngc inpul (oalurc 
pattern vector fAoi corresponding to the same registered person are selected as one or more recognized referential 
lace image patterns (step S314). Thereafter, one or more recognized face images indicated by the recognized refer- 
ential face image patterns are displayed on the display 17 as one or more candidates for the specific person, or the 
recognized referential face image patterns are output to the output terminal 13 (step S3i 5). 

[012D] In this embodiment, a set of referential face image patterns is delerrinined f rom a set of referential identification 
face images obtained by using the image scanner 13. However, because a feature extraction transformation using the 
feature extraction matr ix Fi is performed for each face image pattern lo obtain th gjset of refe rential face image paltems, 
even though a scTo^fcrcnliHl video faco images nro obtained by using lhc video camera 1 2. a scl of rcfercnlial lace 
image patterns can be determined from the set of referential video face images. 

[0121] Also, in this cmbodtmcnl a group of input face images of the specific person arc oblained by using lhc video 
camera 12. However, because a feature extractbn transformation using the ^eature ^igcti9e.g a^ ^ Fi is perform.ed 
^5 tor the input face image patterns, even though the input face image of the sptifllSferson is bcJrSned by using the 
image scanner 1 3, the average input feature pattern vector fAgi independent of a statistic propeny difference between 
the first image pattern obtaining process and the second image pattern obtaining process can be obtained. 
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[0122] Accordingly, because the leature extraction matrix Fi, which minimizes an overlapping area between the pat- 
tern distribution of a face image pattern of one registered person (or a group of face image patterns ot one registered 
person) and the perturbation distribution of the face image pattern, is assumed in advance tor each registered person 
by using a group ot video face images and one identification 1ace image, one referent talface image pattern independent 

B of a statistic property difference between the first image pattern obtaining process using the image scanner 1 3 and the 
second image pattern obtaining process using the video camera 12 can be obtained for each registered person, and 
one or more referential face image patterns most similar to a face image pattern corresponding to a specific person 
can be selected regardless of whether the lace image pattern is obtained according to the first image pattern obtaining 
process or the second image pattern obtaining process. 

TO [0123] In this embodiment the perturbation covariance matrix Cp(i) is calculated according to the equation (17) by 
using tho prillcrn pcrlurbalicn Pji. However il is fipplicabic Ihrjl a pHllcrn pcrlurbfilion Pi be CHlculfjicd by using the 
average pattern vector ai and the perturbation covariance matrix Cp(i) be calculated by using the pattern perturbation 
Pi. That iS: a pattern perturbation Pi ot the i-th registered person is calculated in the pattern perturbation calculating 
means 82 according to a following equation Pi = ai - bi, a perturbation covariance matrix Cp(i) is calculated for each 

y5 registered person in the perturbation covariance assuming means S3 by using the pattern perturbations Pi according 
to a following equation. 

Cpii] = PiPi^ 

20 

[0124] Also, a moving picture composed of a series ot still pictures is obtained for each registered person by directly 
photographing the lace of the registered person with the video camera 12, and one teaching video tace image pattern 
is produced tor each sdll picture. However the third embodiment is not limited to obtain a oKiving picture composed 
of a series of stilt pictures for each registered person. That is, it is applicable that a still picture be obtained for each 
i?5 registered person by directly photographing the face of the registered person with the video camera 1 2 and one teaching 
video face image pattern ai be produced for each registered person. In this case, a pattern perturbation Pi of the i-th 
rcgislcrcd person is CHlculwlcd in Ihc pHlLcrn pcrlurbnUon calculnLing monns 92 ;Hccording lo a following oquHLicn Pi 
= ai - bi, a perturbation covariance matrix Cpfij is calculated tor each registered person in the perturbation covariance 
rissuming mcnns £3 by using the pHllcrn pcrlurbnlions Pi nccording lo h following oquHlion. 

3Q 

Cp(i) = PiPi'^ 

[0125] Also, in this emtxidiment, the referential identification tace image patterns B^i are obtained by scanning the 
55 tace photographs of the registered persons by the image scanner 1 3, and the input face image pattern A^j is obtained 
by directly photographing the specific person with the video camera 12. However, it is applicable that the referential 
idcnlificHtion fncc imago pallorns be oblHincd by directly pholographing Ihc rcgislcrcd persons rjnd Ihc input face 
image pattern be obtained by scanning the face photograph ot the specific person. In this case, a group of referential 
identification tace image patterns A^jl are obtained for each registered person, an average referential identification 
40 lace image pattern A, i ot each registered person is calculated according to a foltowing equation 



L 

All = ( l/L)2Aiji 

a referential feature pattern vector fAii = FiA^i is obtained^ an input face image panern Bo ot the specific person is 
obtained, an input feature pattern vector tBji corresponding to each registered person is calculated according lo a 
50 tollowing equation 

tBgi = FiB2i, 

ss and the average referential identification face image pattern A^ i is collated with the input teature panern vector fBgi lor 
each registered person in the optimum match processing performing means 73. 

[0126] Also, the video camera 1 2 and the image scanner 1 3 are used as image receiving means. However the image 
receiving means is not limited to the video camera 12 and the image scanner 13; and it is applicable that ^voor more 
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image receiving units be used as ihe image receiving means. 

[0127] Also, the set of referential face image patterns is obtained in the otf-iine processing. However, it is applicable 
that a set of referential face image patterns obtained in an external apparatus be stored in advance in the auxiliary 
storing unit 16. 

[0128] Also, the set ot referential face image patterns obtained in the off-line processing is stored in the referential 
face image data base 30. However, it is applicable that the set of referential face image patterns be stci ed in advance 
in the auxiliary storing unit 16. 

[0129] Also, in this embodiment, aface image pattern is recognized in the pattern recognizing apparatus n . However, 
it is applicable that an image pattern of a mobil can parts or the like be recognized in the panern recognizing apparatus 
n. Also, it is applicable that patterned data obtained from audio data, character data or the like be recognized in the 
paltcrn recognizing rjpp^rrilus 11. 

(Fourth Embodiment) 

[0130] In a fourth embodiment, the judgement whether or not a specific person is identical with a photograph person 
corresponding to an identification card exhibited by the specific person is performed in a pattern identity judging ap- 
paratus according to a pattern identity judging method by collating a video face image pattern obtained by directly 
photographing a face of the specific person with a photograph face innage pattern obtained by scanning a face photo- 
graph of the identification card. A hardware of the pattern identity judging apparatus is the same as that of the pattern 
recognizing apparatus 11 shown in Fig. 1 . 

[0131] Fig. 14 is a block diagram showing a functional structure of the CPU 25 arranged in the pattern identity judging 
apparatus 11 according to a fourth embodimeni of the present invention. 
[0132] As shown in Fig. 1 4, the CPU 25 comprises 

a feature extraction calculating means 91 tor calculating a feature extraction matrix F from the teaching video lace 
images and the teaching identification face images; ' 

afeature pattern extracting means 92 for performing a feature extraction transformation using the feature extraction 
matrix F for afirst input face image pattern of a first input face image, whicfi is obtained by directly photographing 
rj face of H specific pcrsoH in Ihc video camcrn 1 2, lo produce 

a first input feature pattern vector ffli^, and performing a feature extraction transformation using the feature extrac- 
tion matrix F for a second input face image pattern bj^ of a second input face image, which is obtained by scanning 
a lace photograph of a photograph person attached to an identification card in the image scanner 13, to produce 
a second input feature pattern vector fbi^; and 

an identity judging means 93 (or estimating a similarity betv/een the first input feature pattern vector faj„ and the 
second input feature pattern vector by collating the first input feature pattern vector faj^ with the second input 
feature pattern vector tbjn and judging according to the similarity whether or not the specific person corresponding 
to the first input feature pattern vector fai„ is identical with the photograph person of the face photograph corre- 
sponding to the second input feature pattern vector fbjn- 

[0133] Fig. 15 is a block diagram of the feature extraction matrix calculating means 91 . 
[0134] As shown in Fig. 15, the feature extraction calculating means 71 comprises 

the pattern covariance assuming means 81 ; 
the patterr^ perturbation calculating means S2: 

a perturbation covariance assuming means 94 for assuming a perturbation covariance matrix Cp(r; of the pattern 
perturbations Pji 0=1 to L) calculated in the panern perturbation calculating means 52 for each registered person 
Hnd CHlcul^jling »n Hvcrngc pcrlurbHlicn covariance malrix Cp from the pcrlurbnlion covririanco mHlriccs Cp(i) ol 
the registered persons; 

the both-diagonalizing matrix calculating means 54 for calculating a both-diagonalizing matrix H; 
the diagonal matrix producing means 55 for diagonalizing the pattern covariance matrix Cs to produce a diagonal 
matrix Ds of the pattern covariance matrix Cs and diagonalizing the perturbation covariance matrix Cp to produce 
a dagoi^l matrix Dp of the perturbation covariance matrix Cp; 

the amplitude re -transformation matrix calculating means 55 for calculating an amplitude re-transformation matrix 
L; and the feature extraction matrix calculating means 57 1or calculating a feature extraction matrix F = UH. 

[01 35] In the above configuration, an operation performed in the pattern identity judging apparatus 11 is described. 
[01 36] Fig. 1 6 shows an operation flow in an off-line processing initially performed in the feature extraction calculating 
means 9l of the pattern identity judging apparatus 11. 
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[0137] As shown in Fig. 16. the pattern covariance matrix Cs and the perturbation covariance matrices Cp(i) are 
calculated in the steps S301 to S504 in the same manner as in the first embodinnent. 

[01 38] Thereafter, an average perturbation covariance matrix Cp is calculated fronn the perturbation covariance ma- 
^ trices Cp(i) o1 the registered persons in the perturbation covariance assuming means 94 according to a Icllowing equa- 
s lion (step S401). 



Cp = (1/M)S Cp(i) (26) 



where M denotes a number of perturbation samples {M<N). 

[0139] Thereatter, a boih-diagonalizing nnatrix H is calculated according to the equation (6) trom the pattern covari- 
^s ance matrix Cs and the average perturbation covariance matrix Cp in the both-diagonalizing matrix calculating means 

54 (step S402], a diagonal matrix Ds o1 the pattern covariance matrix Cs and a diagonal matrix Dp o1 the average 
perturbation covariance matrix Cp are calculated according to the equation (7) in the diagonal matrix producing means 

55 (step S403). an amplitude re-transtormaticn nnatrix L is calculated according lo the equation (3) in the amplitude 
re-transformation matrix calculating means 56 (step S404), and a feature extraction matrix F is calculated according 

^0 to the equation (10) in the feature extraction matrix calculating means 57 (step S405). 

[0140] Fig. 1 7 shows an operation flow in an on-line processing performed in the pattern identity judging apparatus 
according to the fourth embodiment. 

[0141] As shown in Fig. 17, when a specific person exhibits an identification card, a face of the specific person is 
directly photographed with the video camera 1 2 to obtain a rnoving picture composed of a series of still pictures. Each 
25 still picture indicates a first input face image. The group o1 first input face images of the specific person are stored in 
the video face image memory 21 (step S411}. Also, a face photograph of the identificatioi^ card is scanned by the 
image scanner 13 to obtain a second input face image, and the second input face image of the face photograph is 
stored in the identification face image memory 23 (step S411 ). 

[0142] Thereof Icr, h group of first inpul fncc imngc pHltcms ajj of the specific person nrc produced from Ihc group 
00 of first input face images (step S41 2). and a second input face image pattern bj^ of the photograph person is produced 
from the second input face image (step S412), Thereafter, an average o1 the first input face image patterns ayi is 
calculated according to a following equation to obtain a first average input face image patterns a,^ (step S41 3). 



3S L 

am = — " (27) 



[0143] Thereafter, a first input feature pattern vector fai^ of the specific person and a second input feature pattern 
■ vector 1b|„ of the photograph person are calculated acconding to the equations (i 3) and (l 4) in the same manner as 
in the second embodiment (step S41 4). the first input feature pattern vector faj^ is collated with the second input feature 
pattern vector to judge In an on-line processing whether or not the specific person is identical with the photograph 
person (step S41 5). 

4S [0144] In cases where a similarity between the first input feature pattern vector fai„ and the second input feature 
pattern vector fbin is sutticiently high, it is judged that the specific person is identical with the photograph person, and 
H coll;^tion rcsull indicating thnl Ihc specific person is idcnlicHl with Ihc photograph person is dispinycd on ihc display 
16 or is output to the output terminal 18 (step S205). In contrast, in cases where the similarity is not sufficiently high, 
it is judged that the specific person is not identical with the photograph person, and a collaUon result indicating that 

50 the specific person is not identical with the photograph person is displayed on the display 1 6 or is output to the output 
terminal 18 (step 3416). 

[014S] Accordingly, even though the processes for obtaining the tv/o face images differ from each other, the judge- 
ment whether or not the specific person photographed by the video camera 1 2 is identical with the photograph person 
of the face photograph scanned by the image scanner 13 can be correctly performed. That is. in cases where the 
specific person is actually identical with the photograph person of the face photograph, a collation result indicating that 
the specific person is identical with the photograph person of the face photograph can be reliably obtained. 
[0146] Also, even though a moving picture composed of a series o1 still pictures is obtained for each registered 
person, because the average perturbation covariance matrix Cp is calculated by averaging the perturbation covariance 
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matrices Cp(i) o1 the regist'ered persons, the feature extraction matrix F common to the registered persons can be 
obtained, so that the identity o1 a person photographed by the video camera 12 with a person expressed by a tace 
photograph can be judged in the same manner as in the second embodiment. 

[0147] In this embodiment, the video camera 12 and the image scanner 13 are used as image receiving means. 
However, the image receiving means is not limited to the video camera 1 2 and the inaage scanner 1 3, and it is applicable 
that two or mot e image receiving units be used as the image receiving means. 

[0148] Also, the4et of teaching face image patterns is obtained in the off-line processing. However, it is applicable 
that a set ol teaching face image patterns obtained in an external apparatus be stored in advance in the auxiliary storing 
unit 15. / 

[0149] Also, the set of teaching face image patterns obtained in the off-line processing is stored in the relerentia! 
Ihcc imngc dnLa bHsc-30,. However, il is applicable thai Ihc scl of tCHching face image prillcrns be stored in ndvancc 
in the auxiliary storing unit 1 6. 

[0150] Also, in this embodiment, twotace image patterns are collated with each other in the pattern identity judging 
apparatus 11. However, it is applicable that image patterns ol a mobil car parts or the like be collated v;ith each other 
in the pattern identity judging apparatus 1 1 . Also, it is applicable that panemed data obtained from audio data, character 
data or the like be collated with each other in the pattern identity judging apparatus 11 . 

[0151] Having illustrated and described the principles of the present invention in a preferred embodiment thereof , it 
should be readily apparent to those skilled in the art that the invention can be modified in arrangement and detail 
without departing from such principles. We claim all modifications coming within the scope of the accompanying claims. 



Claims 

1. A pattern recognizing method, comprising the steps o1: 

obtaining a set of first teaching patterns of a plurality of teaching samples according to a first pattern obtaining 
process; 

obtaining a set of second teaching patterns of the teaching samples according to a second pattern obtaining 
process differing from Ihc firsl pHllcrn obtaining process; 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second teaching 
patterns; 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set ol 
second teaching patterns; 

calculating a feature extraction matrix, which minimizes an overlapping area between the teaching pattern 
distribution and the teaching perturbation distribution, from the teaching pattern distribution and the teaching 
perturbation distribution; 

obtaining a set of referential patterns of a plurality of referential samples according to the first pattern obtaining 
process; 

calculating a set of referential feature patterns of the referential samples from the set ot referential patterns 
according to the feature extraction matrix, the set of referential feature patterns being independent of the first 
pattern obtaining process and the second pattern obtaining process; 

receiving an input pattern of an input sample according to the second pattern obtaining process; 
calculating an input feature pattern of the input sample from the input pattern according to the feature extraction 
matrix; 

selecting a specific referential feature pattern most similar to the input feature pattern from the set of referential 
feature patterns; and 

rccogni/ing h specific rcfercnlial sample corresponding lo Ihc specific rcfcrcnlial feature pallcrn as the inpul 
sample. 

2. A pattern recognizing method according to claim 1 in which the step of calculating a teaching pattern distribution 
comprises the steps of 

calculating a teaching pattern covariance matrix of a pattern sample space from the set of first teaching patterns 
or the set of second teaching patterns; and 

assuming the teaching pattern covariance matrix as the teaching pattern distribution, 

the step of calculating a teaching distribution of a perturbation comprises the steps of 

calculating a teaching pattern perturbation between one first leaching pattern of one teaching sample and one 

second teaching pattern of the teaching sample for each teaching sample; 
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calculating a teaching perturbation ccvarianca matrix from the leaching pattern penurbations of the teaching 
samples; and 

assuming the teaching perturbation covariance matrix as the teaching perlurbation distribution, and 
the step of calculating a feature extraction matrix comprises the steps c1 

calculating a both-diagonalizing matrix, which diagonalizes both the teaching pattern covariance matrix and 
the teaching perturbation covariance matrix, from the teaching pattern covariance matrix and the teaching 
perturbation covariance matrix; 

diagonalizing the teaching patterti covariance matrix according to the both-diagonalizing matrix to produce a 
diagonal matrix of the teaching pattern covariance matrix: 

diagonalizing the teaching perturbation covariance matrix according to the both-diagonalizing matrix to pro- 
duce a dUgon»\ mHLrix of Ihc leaching perlurbation covariance mnLrix: 

calculating an amplitude re-transformalion matrix, which again transforms a rsterentlal pattern covariance 
matrix indicated by the set of referential teature patterns to adjust amplitudes oi diagonal elements of the 
referential pattern covariance matrix after the referential pattern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized, from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix and the amplitude re-transformation 
matrix. 

A pattern recognizing method according to claim 2 in which the teaching pattern covariance matrix Cs is expressed 
in an equation 

CS = (1/N)2 (fax - m) (hi - m)" 
i = l 

where bi denotes one first teaching pattern or one second teaching pattern o1 an i-th teaching sample, N denotes 
Iho number of leeching samples, and m dcnolcs nn rjvcrHgc p^illcrn of the set of first IcHching p«lLcrns or the scl 
o1 second teaching pattern. 

the teaching pattern perturbation Pi is expressed in an equation 

Pi at - bi 

where ai and bi denotes one first teaching pattern and one second teaching pattern of ag i-th teaching sample. 
Iho IcHching pcrturbHlion covHriancc mhiUix Cp is expressed in nn cquaiion 

M 

Cp = (l/M)2PiPi'^* 
1 = 1 

where M denotes Ihe number of pcrlurbMlion snmplcs sclccicd from Ihc IcHching s^implcs. 
the both-diagonalizing matrix H is expressed in an equation 

T -1/2 T 
H = M' A 0) 

Where O denotes an eigenvector matrix of the teaching pattern covariance matrix Cs^ A denotes an eigenvalue 
matrix ol the teaching pattern covariance matrix Cs, 'V denotes an eigenvector matrix of a matrix K defined in an 
equation 

and = diap c^-'^'^., (Tg'^'^. —1 is satisfied for the diagonal matrix A = diag fa,; ct^, 
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the diagonal matrix Ds of the teaching pattern covari since mairix Cs is expressed in an equation 

Ds^HCoh''", 

the diagonal matrix Dp of ihe teaching pertutbation covaiiance matrix Cp is expressed in an equation 

Dp = HCpH^= 

the amplitude re-Lransforrralion matrix L is expressed in an equation 

Lading [(>V''^ ^ X/^^-^'^". (X^t^-" ^ H 
where n denotes an arbitral real nunnber, the feature extraction matrix F is expressed in an equation 

F=:LH. 

each referential feature pattern fbi is expressed as a product of F and bi in an equation 

tbi = Fbi 

where bi denotes a vector of one referential pattern, and the input feature pattern fa^ is expressed as a product 
of F and in an equaticr> 

fa^ = 

where denotes a vector of the input pattern. 

A pattern recognizing method according to claim 1 in which the step of calculating a teaching distribution of a 
perturbation comprises the step of 

calculating a teaching perturbation distribution betsveen one first teaching pattern of one teaching sample and 
one second teaching pattern of the teaching sample for each teaching sample, 
the step of calculating a feature extraction matrix comprises the step of 

c«lculr»ling h fcHturc oxlrnclion mnlrix, which miniml/cs an overlapping nrcH bolwccn a leaching pallcrn dis- 
tribution of one teaching sample and the teaching perturbation distribution of the teaching sample, from the 
teaching pattern distribution and the teaching perturi^ation distribution of the teaching sample for each teaching 
sample, 

the step of obtaining a set of referential patterns comprises the step of 

obtaining a set of referential patterns of the teaching samples according to the lirst pattern obtaining process 
or the second pattern obtaining process, 

the step of calculating a set of referential feature patterns comprises the step of 

calculaling ono rclorcnlial fcalurc pallcrn of ono leaching sampio from one rcfcrcnlial pallcrn of Lho leaching 
sample according to the feature extraction matrix of the teaching sample for each teaching sample, 
the step of calculating an input feature pattern comprises the step of. 

calculating an input feature pattern of the input sample from the input pattern according toihe feature extraction 
matrix of one teaching sample tor each teaching sample, and the step of selecting a specific referential feature 
pattern comprises the steps o1 

estimating a similarity between one input feature pattern corresponding to one teaching sample and one ref- 
erential feature pattern of the same teaching sample; and 

selecting a specific referential feature pattern of a specific teaching sample most similar to the input feature 
pattern corresponding to the teaching sample from the set of referential feature patterns. 
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A pattern recognizing method according to claim 4 in which the step of caiculaling a teaching pattern distribution 
comprises the steps of 

calculating a teaching pattern covariance matrix ot a pattern sample space from the set o1 first teaching patterns 
or the set of second teaching patterns: and 

assuming the teaching pattern covariance matrix as the teaching pattern distribution, 
the step of calculating a teaching perturbation distribution comprises the steps ot 

calculating a teaching pattern perturbation between one first teaching pattern of one teaching sample and one 
second teaching pattern of the teaching sample tor each teaching sample; 

calculating a teaching perturbation covariance matrix from one teaching pattern perturbation of one teaching 
sample for each LcHching sample; and 

assuming the teaching perturbation covariance matrix as the teaching perturbation distribution tor each teach- 
ing sample, and 

the step of calculating a feature extraction matrix comprises the steps of 

calculating a both-diagonalizing matrix, which diagonalizes both the teaching partem covariance matrix and 
the teaching perturbation covariance matrix of one teaching sample, from the teaching pattern covariance 
matrix and the teaching perturbation covariance matrix for each teaching sample; 

diagonaliztng the teaching pattern covariance matrix according to the both-diagonalizing matrix of each teach- 
ing sample to produce a diagonal matrix of the teaching pattern covariance matrix corresponding to one teach- 
ing sample for each teaching sample; 

diagonaiizing the leaching perturt^ation covariance matrix of one teaching sample according to the both-diag- 
onalizing matrix o1 the teaching sample for each teaching sample to produce a diagonal matrix ol the leaching 
pertu rbation covariance matrix tor each teaching sample: 

calculating an amplitude re-transformation iriatrix. which again transforms a referential pattern covariance 
matrix indicated by the set of referential feature patterns to adjust amplitudes o1 diagonal elements of the 
referential pattern covariaiKe rnatr ix after the referential pattern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized, from the diagonal nnatrices corresponding to one teaching sample for 
each teaching sample; and 

caiculaling Ihc fcalurc cxlraction matrix corresponding to one leaching sample from the bolh-diagonali/ing 
matrix corresponding to the teaching sample and the amplitude re-transfomnaiion matrix corresponding to the 
teaching sample for each teaching sample. 

A pattern recognizing method according to claim 5 in which the teaching pattern covariance matrix Cs is expressed 
in an equation 

w 

Cs - - in)(bi - m)T 

i = l 

where bi denotes one first teaching pattern or one second teaching pattern ot an i-th teaching sample, N denotes 
the number of teaching samples, and m denotes an average pattern of the set of first teaching patterns or the set 
of second teaching pattern. 

the teaching pattern perturbation Pi is expressed in an equation 

Pi = ai - bi 

where ai and bi denotes one first teaching pattern and one second teaching pattern of an i-th teaching sample, 
the teaching perturbation covariance matrix Cp(i) corresponding to each teaching sample is expressed in an equa- 
tion 

Cp(i) = PiPi"^, 

the both-diagonalizing matri:< Hi corresponding to each teaching sample is expressed in an equation 
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Hi = ^V^.\''^<X>^ 

where <D denotes an eigenvector matrix of the leaching pattern covariance nnatrix Cs, A denotes an eigenvalue 
matrix of the teaching pattern covariance matrix Cs. H' denotes an eigenvector matrix of a nnatrix K defined in an 
equation 

-i.*? T -1'2 T T 

and 

A-i/^ = diag {a,i-W2, 0^,-^/2, 
is satisfied for the diagonal nnatrix 

A = diag [a,i, a^2. ] , 

the diagonal matrix Ds(i) of the teaching pattern covariance matrix Cs corresponding to each teaching sample is 
expressed in an equation 

Ds(i)= HiCsHi^= 

the diagonal matrix Dp(t) of the teaching perturbation covariance matrix Cp(i) corresponding to each teaching 
sample is expressed in an equation 

Dp(i) = HiCp(i)Hi^, 

the amplitude re -transformation matrix Li corresponding to each teaching sample is expressed in an equation 
Li = diag X.^^pja) -l/2n, {X^^*^'^ X^jCPlnj-i^^n, ] 

where n denotes an arbitral real number, Ihe feature extraction matrix Fi corresponding to each teaching sample 
is expressed in an equation 

Fi = LiHi, 

Iho rcfcrcnliHl fcHluro pHltorn fbi corresponding to o;ich leaching sample is expressed hs h produci of Fi wnd bi in 
an equation 

tbi = Fibi 

where bi denotes a vector of one referential pattern, and the input feature pattern faai corresponding to each 
teaching sample is expressed as a product 0I Fi and a2 in an equation 

faji = Fiaj 
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where 82 denotes a vectcr o\ the input pattern. 

A pattern recognizing method according to claim 1 in '.vhich the step of obtaining a set 0I second teaching patterns 
comprises the step of 

obtaining a group ol second teaching patterns for each leaching sample, 

the step of calculating a teaching distribution ol a perturbation comprises the step of 

calculating a teaching perturbation distribution between one first teaching pattern ol one teaching sample and 
one group of second teaching patterns of the teaching sample tor each teaching sample, 
the step of calculating a feature extraction matrix comprises the step cf 

CHlcuIaltng h IcHiurc cxIrHClion rralrix, v/hich minimizes an ovcrlrtpping nrca bolwccn h ICHching pHllcrn dis- 
tribution of one teaching sample and the teaching perturbation distribution of the teaching sample, fronn the 
teaching pattern distribution and the teaching perturbation distribution of the teaching sample tor each teaching 
sample, 

the step of obtaining a set of referential patterns comprises the step of 

obtaining a set of referential patterns of the teaching sample according to the first pattern obtaining process, 
the step of receiving an input pattern comprises the step of 

obtaining a group of input patterns of the input sample according to the second pattern obtaining process^ the 
step of calculating an input feature pattern comprises the steps of 

calculating a group of input feature patterns of the input sample from the group of input patterns according to 
the feature extraction matrix of One teaching sample for each teaching sample; and 

calculating an average input feature pattern from the group of irtput feature patterns for each teaching sample, 
and the step of selecting a specific referential feature pattern comprises the steps of 

estimating a simiterity between one average input feature pattern corresponding to one teaching sample and 
one average referential feature pattern of the same teaching sample; and selecting a specific average refer- 
ential feature pattern of a specific teaching sample most similar to the average input feature pattern corre- 
sponding to the teaching sample from the set of average referential feature patterns. 

A pHtlcrn recognizing method ficcording lo claim 7 in which Ihc slop of cfdlculnling h IcHching pHttcm distribution 
comprises the steps of 

calculating a teaching pattern covariance matrix of a pattern sample space from the set of first teaching patterns 
or the set of second teaching patterns; and 

assuming the teaching pattern covariance matrix as the teaching pattern distribution, 
the step of calculating a teaching perturbation distribution comprises the steps of 

calculating a group of teaching pattern perturbations betv/een one first teaching partem of one teaching sample 
and one group of second teaching patterns of the teaching sample for each teaching sample; 
calculating a teaching perturbation covariance matrix from one group of teaching pattern perturbations of one 
teaching sample for each leaching sample; and 

assuming the teaching perturbation covariance matrix as the teaching perturbation distribution for each teach- 
ing sample, and 

the step of calculating a feature extraction matrix comprises the steps of 

calculating a both -diagonal i zing matrix, which diagonalizes both the teaching pattem covariance matrix and 
the teaching perturbation covariance matrix of one teaching sample, from the teaching pattern covariance 
matrix and the teaching perturbation covariance matrix for each teaching sample; 

diagonalizing the teaching pattern covariance matrix according to the both-diagonal izing matrix of each teach- 
ing srimpic to produce h dingonrjl mnlrix ol the ICHching pHllcrn covHri^incc malrix corresponding lo ono IcHch- 
ing sample for each teaching sample; 

diagonalizing the leaching perturbation covariance matrix of one teaching sample according lo the both-deg- 
onaiizing matrix of the teaching sample for each teaching sample to produce a diagonal matrix ol the teaching 
perturbation covariance matrix for each teaching sample; 

calculating an amplitude le-transformation matrix, which again transforms a referential pattern covariai^ce 
matrix indicated by the set of referential feature patterns to adjust amplitudes of diagonal elements of the 
referential pattern covariance matrix alter the referential panern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized, from the diagonal matrices corresponding to one teaching sample for 
each teaching sample; and 

calculating the feature extraction matrix corresponding to one teaching ^mpie from the both -diagonal izing 
matrix corresponding to the teaching sample and the amplitude re-transformation matrix corresponding to the 
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teaching sample tor each teaching SEimple. 

A pattern recognizing method according to claim 8 in which the leaching pattern covariance matri;< Cs is expressed 
in an equation 

K 

= (1/N)2(fai - in)(bi - m)? 

i = l 

where bi denotes one first teaching pattern or one second teaching panern of an i-th teaching sample, N denotes 
the number of teaching samples, and m denotes an average pattern of the set cf tirst teaching patterns or the set 
o1 second teaching paitem, 

each teaching pattem perturbation Pji is expressed in an equation 

Pji _ aji - bi (i_l to N, and |-1 to L) 

where aji denotes one second teaching pattem of an i-th teaching sample. L denotes the number of second teaching 
patterns in one group corresponding to one teaching sample and bi denotes one first teaching pattern of an i-th 
sample, the teaching perturbation covariance matrix Cpfi) corresponding to each teaching sample is expressed 
in an equation 

L 

cp{i) = (1/L)5; PjiP jil', 

j = l 

the both-diagonalizing matrix Hi corresponding to each teaching sample is expressed in an equation 

T •1.-"' T 

Hi = H' A 

Where <b denotes an eigenvector matrix of the teaching pattern covariance matrix Cs, A denotes an eigenvalue 
matrix of the teaching pattern covariance matrix Cs, denotes an eigenvector matrix of a matrix K defined in an 
cqu;ilion 

K = (A '*^4)')Cp(i)(A ) 

and 

is satisfied for the diagonal matrix 

A = diag ( a^^, a^,, J , 

the diagonal matrix Ds(i) of the leaching pattern covariance matrix Cs corresponding to each teaching sample is 
expressed in an equation 

Ds(i)= HiCsHi"^. 
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the diagonal matrix Dp(i) of the teaching perturbation covariance matrix Cp(i) corresponding to each teach ing 
sample is expressed in an equaUon 

Dp(i) = HiCp(i)Hi'^, 

the amplitude re-transfcmnation matrix Li corresponding to each teaching sample is expressed in an equation 



Li = diag [(Xi^t^:" + X,^<P)r. ) -l/2n^ (/.l2^^:" + X^^iP):^) -l/ln ^ ; 

where n denotes an arbitral real number the leature extraction matrix Fi corresponding to each teaching sample 
Is expressed In an equation 

Fi = UHi 

the referential feature pattern fB^i corresponding to each teaching sannple is expressed as a product of Fi and i 
in an equation 

fB,t = Fi3Ti 

where B^i denotes a vector of one referential pattern, the group of input leature patterns (Agji corresponding to 
CHch leaching snmplc is gk pressed as h product of Fi nnd A2j in nn cqu;^lion 

1A2ji = FiAgj (j=1 toL) 

\whcro Agj dcnolcs h vector of CHch input pHltcrn. and L denotes the number of input pHltcrns. nnd 
the average input teature pattern fAjj is expressed In an equation 

L 

fA.i = (l/DlifA.ji 

10. A pattern recognizing apparatus, comprising; 

first pattern obtaining means for obtaining a set of first teaching patterns of a pluraliry ol teaching samples 
according to a first pattern obtaining process; 

second pattern obtaining mear^s lor obtaining a set ol second leaching patterns of the leaching samples ac- 
cording to a second pattern obtaining process differing from the first pattern obtaining process: 
feature extracting means for calcutaiirig a teaching pattern distribution Ircm the set of first teaching patterns 
obtained by the first pattern obtaining means or the set of second teaching patterns obtained by the second 
pattern obtaining meanS: calculating a leaching distribution of a perturbation between the set of first leaching 
pHllcrns and the scl ol second tcHching paltcrns, nnd calculnLing n fcnLurc cxtrnclion nrwlrix, which minimi/cs 
an overlapping area between the teaching pattern distribution and the teaching perturbation distribution, from 
the teaching pattern distribution and the teaching perturbation distribution; 

referential feature pattern calculating means for obtaining a set of referential patterns of a plurality of referential 
samples accofxding to the first pattern obtaining process, and calculating a set of referential feature patterns 
of the referential samples from the set of leferential patterns according to the feature extraction inatrix calcu- 
lated by the feature extracting means to make the set ol referential feature patterns independent ot the first 
pattern obtaining process and the second pattern obtaining process; and 

input pHllcrn rocogni/ing mcHns for receiving an inpul pHllcrn of an input sample according lo Ihc second 
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pattern cbtfiining process, catculaiing an input feature panern of the input sample from the input pattern ac- 
cording to the feature extraction nnatrix calculated by the -feature extracting means, selecting a specific refer- 
ential feature pattern nncst similar to the input feature pattern from the set of referential feature patterns cal- 
culated by the referential feature pattern calculating nneans, and recognizing a specific referential sample 
5 corresponding to the specific referential feature pattern as the input sample. 

11. A pattern recognizing apparatus according to claim 10 in which the feature extracting means comprises 

pattern covariance assuming means for calculating a teaching pattern covariance matrix of a pattern sample 
space from the first teaching patterns or the second teaching patterns and assuming the teaching pattern 
covrtriHncc mnlrix as Ihc loaching pHllcrn dislribuUon; 

pattern perturbation calculating means lor calculating a teaching pattern perturbation between one first teach- 
ing pattern of one teaching sample and on© second leaching pattern of the teaching sample for each teaching 
sample; 

perturbation covariance assuming means for assuming a teaching perturbation covariance matrix from the 
teaching pattern perturbations calculated by the pattern perturbation calculating means as the teaching per- 
turbation distribution: 

both-diagonalizing matrix calculating means for calculating a both-diagonalizing matrix, v^hich diagcnalizes 
both the teaching pattern covariance matrix assumed by the pattern covariance assuming means and the 
teaching perturbation covariance matrix assumed by the perturbation covariance assuming means, from the 
teaching pattern covariance matrix and the teaching perturbation covariance matrix; 

diagonal matrix producing means for diagonatizing the teaching pattern covariance matrbc assumed by the 
pattern covariance assuming means according to the both-diagonalizing matrix calculated by the both-diag- 
Of^aiizing matrtx calculating means to produce a diagonal matrix of the teaching pattern covariance matrix, 
and diagonalizing the teaching perturbation covariance matrix assumed by the perturbation covariance as- 
suming means according to the both-diagonalizing matrix to produce a diagonal matrbc of the teaching per- 
turbation covariance nnatrix: 

amplitude re-transformation matrix calculating means for calculating an amplitude re-transformation matrix, 
v/hich fignin IrHnslcmns h rcrcrcnli«l pHLlcrn covariance nrvilrix indtcHlcd by Iho set of rclcrcnliHi fcHLurc pwl- 
terns to be calculated by the referential feature pattern calculating means to adjust amplitudes o1 diagonal 
elements of the referential pattern covariance matrix after the referential pattern covariance matrix is trans- 
formed by the both-diagonalizing matrix calculated by the both-diagonalizing matrix calculating means to be 
diagonalized, from the diagonal matrices; and 

calculating the feature extract ion matrix from the both-diagonalizing matrix calculatedby the both-diagonalizing 
matrix calculating means and the amplitude re-transformaticn matrix calculated by the amplitude re-transfor- 
mation matnx calculating means. 

12. A pattern recognizing apparatus, comprising: 

40 first pattern obtaining means for obtaining a set of first teaching patterns of a plurality of registered samples 

according to a first pattern obtaining process; 

second pattern obtaining means for obtaining a set of second teaching patterns of the registered samples 

according to a second pattern obtaining process differing from the first pattern obtaining process; 

feature extracting means for calculating a teaching pattern distribution from the first teachirig patterns obtained 

4B by the first pattern obtaining means or the second teaching patterns obtained by the second pattern obtaining 

means, calculating a teaching distribution of a perturbation between one first teaching pattern of one registered 
snmplc and one second IcHchlng pHLlcrn of Ihc registered snmplc for CMch rcgislcrcd snmplc. nnd calcuinLing 
a feature extraction matrix, which minimizes an overlapping area between the teaching pattern distribution of 
one registered sample and the teaching perturbation distribution of the registered sample, from the teaching 

50 pattern distribution and the teaching perturbation distribution for each registered sample; 

referential feature pattern calculating means for obtaining a set of referential patterns of the registered samples 
according to the Hrst pattern obtaining process, and calculating a referential featur e pattern of one registered 
sample from one referential pattern of the registered sample according to the feature extraction matrix of the 
registered sample calculated by the feature extracting means for each registered sample to maKe each refer- 

55 enlial feature pattern independent of the first pattern obtaining process and the second pattern obtaining proc- 

ess; and 

input pattern recognizing means for receiving an input pattern of an input sample according to the second 
pattern obtaining process, calculating an input feature pattern corresponding to one registered sample from 
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the input pattern according to th.e feature extraction matrix of the registered sample calculated by the feature 
extracting means lor each registered sample, estimating a similarity between one relerencial feature pattern 
of one registered sample and the input feature pattern correspcnding to the registered sample for each reg- 
istered sample, selecting a specific referential feature pattern most similar to the input feature panern from 
the reterentialfeature patterns calculated by the referential feature pattern calculating means, and recognizing 
a specific registered sample corresponding to the specific refereritial teature pattern as the input sample. 

3, A pattern recognizing apparatus according to claim 12 in which the feature extracting means comprises 

pattern ccvariance assuming means for calculating a teaching pattern ccvariance matrix of a pattern sample 
spHco from the first teaching priLtcrns or Iho second lo^iching pHllcrns rind Hssuming the Icriching pallcrn 
covariance matrix as the leaching pattern distribution; 

pattern perturbation calculating means lor calculating a teaching pattern perturbation between one first teach- 
ing pattern o1 one registered sample and one second teaching pattern of the registered sample for each reg- 
istered sample; 

perturbation covariance assuming means for calculating a teaching perturbation covariance matrix from one 
teaching pattern perturbation calculated by the pattern perturbation calculating means for each registered 
person and assuming each teaching perturbation covariance matrix as the teaching perturbation cistribution; 
bolh-diagonalizing matrix calculating means for calculating a both-diagonalizinp matrix, which diagonalizes 
both the teaching pattern ccvariance matrix assumed by the pattern covariance assuming means and the 
teaching perturbation covariance matrix assumed by the perturbation covariance assuming means, from the 
teaching pattern covariance matrix and the teaching perturbation covariance matri^^ for each registered person; 
diagonal matrix producing means for diagonal i zing the teaching pattern covariance matrix assumed by the 
pattern covariance assuming means according to the both -diagonal izing matrix calculated by the both-diag- 
onalizing matrix calculating means for each registered person to produce a diagonal matrix of the teaching 
pattern covariance matrix for each registered person, and dtagonalizing the teaching perturbation covariance 
matrix assumed by the perturbation covariance assuming means according to the both-diagonal izing matrix 
for each registered person to produce a diagonal matrix of the teaching perturbation covariance matrix for 
CMch rcgislcrcd person; 

amplitude re-transformation matrix calculating means for calculating an amplitude re -transformation matrix, 
which again transforms a referential pattern covariance matrix indicated by the set of referential feature pat- 
terns to be calculated by the referential feature pattern calculating means to adjust amplitudes of diagonal 
elements of the referential pattern covariance matrix after the referential pattern covariance matrix is trans- 
formed by the both-diagonalizing matrix calculated by the both-diagonalizing matrix calculating means to be 
diagonalized, from the diagonal matrices corresponding one registered sample for each registered person; and 
calculating the feature extraction matrixf rem the both-diagonalizing matrix calculated by the both-diagonalizing 
matrix calculating means and the amplitude re -transformation matrix calculated by the amplitude re-transfor- 
malion matrix calculating means for each registered person. 

14. A pattern Identity judging method, comprising the steps of: 

obtaining a set of first teaching pattei nsf rom a plurality of teaching samples accordingio a first pattern obtaining 
process; 

obtaining a set of second teaching patterns from the teaching samples according to a second pattern obtaining 
process differing from the first pattern obtaining process: 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second teaching 
pHllcrns; 

calculating a teaching distribution of a perturbation between the set of first leaching panerns and the set of 
second teaching patterns; 

calculating a feature extraction matrix, which minimizes an overlapping area between the leaching pattern 
distribution and the teaching perturbation distribution, from the teaching pattern distribution and the teaching 
perturtation distribution: 

receiving a first input pattern of a first input sample according to the first pattern obtaining process; 
calculating a first input feature pattern of the first input sample from the first input pattern according to the 
teature extraction matrix, the first input feature pattern being independent of the first pattern obtaining process 
and the second pattern obtaining process; 

receiving a second input pattern of a second input sample according to the second pattern obtaining process; 
calculating a second input teature pattern of the second input sample trom the second input pattern according 



36 



EP 0 944 018 A2 



to the feature extraciion matrix, the second input feature pattern beinq independent of the first pattern obtaining 
process and the second pattern obtaining process; 

collating the tirst input feature pattern with the second input leature pattern to estimate a similarity between 
the first input sample and the second input sample; and 

judging that the first i nput sample is identical with the second input sample in cases where the similarity is high. 

15. A pattern identity judging method according to claim 14 in ^Arfiich the step of calculating a teaching pattern distri- 
bution comprises the step o1 

assuming a teaching panem covariance matrix of a pattern sample space as the teaching pattern distribution, 
Ihc step of calculHling ;-i IcHching dislribulion o( h pcrlurbHlion comprises Ihc steps of 
calculating a teaching pattern perturbation between one tirst teaching pattern of one teaching sample and one 
second teaching pattern of the leaching sample tor each teaching sample; and 

assuming a teaching penurbation covariance matrix from the teaching pattern perturbations as the teaching 
perturbation distribution, and 

the step of calculating a feature extraction matrix comprises the steps o1 

calculating a both-diagonalizing matrix, which diagortalizes both the teaching pattern covariance matrix and 
the teaching penurbation covariance matrix, from the teaching pattern covariance matrix and the teaching 
perturtation covariance matrix; 

diagonalizing the teaching pattern covariance matrix according to the both-diagonalizing nnatrixto produce a 
diagonal matrix of the teaching pattern covariance matrix: 

diagonalizing the teaching perturbation covariance matrix according to the both-diagonalizing matrix to pro- 
duce a diagonal matrix of the teaching perturbation covariance matrix: 

calculating an amplitude le-tiansformation matrix, which again transforms a referential pattern covariance 
matrix indicated by the set of referential feature patterns to adjust amplitudes o1 diagonal elements of the 
referential pattern covariance fnalrix after the referential pattern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized: from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix and the amplitude re-transformation 
mn Irix. 

16. A pattem identity judging method according to claim 15 in which 

the teaching pattern covariance matrixes is expressed in an equation 



H 

Cs = (1/N)2(bi - m](bi - m)^ 

where bt denotes one first teaching pattern or one second teaching pattern of an i-th teaching sample, N 
denotes the number of teaching samples, and m denotes an average pattern of the set of first teaching patterns 
or the set of second teaching pattern, 

the teaching pattern perturbation Pi is expressed in an equation 

Pi = ai - bi 

where ai and bi denotes one first teaching pattern and one second teaching pattern of an i-th teaching sample, 
the teaching perturbation covariance matrix Cp is expressed in an equation 

K 

cp = { 1/M)2 PiPiT 
i=l 

where M denotes the number of perturbation samples selected from the teaching samples. 
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the both-diagonalizing matrix H is expressed in an equation 

where <!> denotes an eigenvector matrix ot the teaching pattern covartance matrix Cs, ,\ denotes an eigenvalue 
matrix ot the teaching pattern covariance rnatrix Cs, denotes an eigenvector matrix o1 a rratrix K defined in 
an equation 

and :\-^^ - diag Icf'''^, a^^^, --] is satisfied tor the diagonal matrix A = diag [c^, a^^ — ]: 

the diagonal matrix Ds of the teaching pattern covariance matrix Cs is expressed in an equation 

Ds = HCsh"^. 

the diagonal matrix Dp of the teaching petturbation covariance matrix Cp is expressed in an equation 

Dp = HCph"^, 

the amplitude re-transformation matrix L is expressed in an equation 

i f/-k "S)n ^ ijj}n.-1/2n {s\r\ - (p)n. -1/'2n , 

v^here n denotes an arbitral real number. 

Ihc fc^ilurc cxlrHCtion mHlriK F is expressed in ein cquHlion 

F = LK 

the first input feature pattern fOjn is expressed as a product of F and aj„ in an equation 

where aj„ denotes a vector of the firsft input pattern^ and the second input feature pattern fbjn is expressed as 
a product of F and bj^ in an equation 

Where denotes a vector ot the second input pattern, 

A pattern identity judging method according to claim 14 in which 

the step of obtaining a set of second teaching panerns comprises the step of 

obtaining a group of second teaching patterns for each teaching sample, 
the step of calculating a teaching pattern distribution comprises the steps of 

c^^lculHling h Lc;^chin9 p«Ucrn covarinnco matrix of a pallcrn SHmpIc spHcc from the sol of firsl leaching pHtlcrns 
or the set of second teaching patterns; and 

assuming the teaching pattern covariance matrix as the teaching pattern distribution, 

the step of calculating a teaching distribution ol a perturbation comprises the steps o1 

calculating a group of teaching panem perturbations between one first teaching pattern of one teaching sample 

and one group of second teaching patterns of the teaching sample for each teaching sample; 
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calculating a teaching perturbation covariance matrix Irom one group of teaching pattern penurbations o1 one 
teaching sample lor each teaching sample; 

calculating an average teaching perturbation covariance matrix from the leaching perturbation covariance 
matrices; and 

assunning the average teaching perturbation covariance matrix as the teaching perturbation distribution, 
the step of calculating a feature extraction matrix comprises the steps ot 

calculating a both-diagonalizing matrix, which diagonalizes both the teaching pattern covariance matrix and 
the teaching peiiurbation covariance matrix, from the teaching pattern covariance matrix and the teaching 
perturbation covaricince rmtrix: 

diagonalizing the teaching pattern covariance matrix according to the boih-diagonalizing matrix to produce a 
dingonHl mriljix of Ihc Ic^iching pfillcrn covriri^nco mnlrix; 

diagonalizing the teaching perturbation covariance matrix according to the both-diagonatizing matrix to pro- 
duce a diagonal matrix of the teaching perturbation covariance matrix: 

calculating an amplitude re-iransformation matrix, which again transforms a referential pattern covariance 
matrix indicated by The set of relerential feature patterns to adjust amplitudes of diagonal elements of the 
referential pattern covariance rrtatrix after the referential pattern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized, from the diagonal matrices; and 

calculating the feature extract icn matrix from the both-diagonalizing matrix and the amplitude re-transfornnation 
matrix, 

the step of receiving a second input pattern comprises the step of 
obtaining a group of second input patterns of the second input sample, and 
the step of calculating a second input feature pattern comprises the steps of 

calculating an average of the group of second input patterns as a second input pattern newfy defined: and 
calculating a second input feature pattern of the second input sample from the second input pattern newly 
defined^ 

A pattern identity judging method according to claim 17 in which 

the teaching pattern covariance matrix Cs is expressed in an equation 



CS = ( 1/N)2 - m) (bi - m)- 



where bi denotes one first teaching pattern of an i-th teaching sample. N denotes the number ol teaching samples, 
and m denotes an average pattern of the set of first teaching patterns, 
each teaching pattern perturbation Pji is expressed in an equation 

Pji = aji - bi (i=1 to N, and i=1 to L) 

where aji denotes one second teaching pattern of an i-th teaching sample: L denotes the number of second teaching 
patterns in one group corresponding to one teaching sample and bi denotes one first teaching pattern of an i-th 
IcHching SMnnplc. 

the teaching perturbation covariance matrix Cp(i) corresponding to each teaching sample is expressed in an equa- 
tion 



L 

Cp(i)* = (1/L)2P jiP jiT, 
the average teaching perturbation covariance matrix Cp is expressed in an equation 
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M 

Cp = (l/M)i;Cp(i) 

6 

where M denotes the number of perturbation sannples selected from the teaching samples, 
the both-diagonalizing matrix H is expressed in an equation 

T T 
H = n' A <P 

Where <I) denotes an eigenvector matrix of the teaching pattern covariance matrix Cs, A denotes an eigenvalue 
matrix ol the teaching pattern covariance matrix Cs, H' denotes an eigenvector matrix of a matrix K defined in an 
75 equation 

K=(A )Cp(A''"<P)')V 

20 and A'^-^ diag [<^^'^''^.. oV^'"'^' ""! '® satisfied for the diagonal matrix A diag [o,. c^, ™], 

the diagonal matrix Ds of the teaching pattern covariance matrix Cs is expressed in an equation 

Ds=HC3H^, 

2$ 

the diagonal matrix Dp of the average teaching perturbation covariance matrix Cp is expressed in an equation 

Dp HCpH"^, 

30 

the amplitude re-transformation matrix L is expressed in an equation 

L = diag [(X/ + ) , (>.2 >-2 ) . — ] 

3$ 

where n denotes an arbitral real number, the feature extraction matrix F is expressed in an equation 

F= LH. 

40 

the first input feature pattern Ibj^ Is expressed as a product of F and in an equation 

Where bj^ denotes a vector ol the first input pattern, the average of the group ot second input feature patterns 

so ai, = (i/L)2ai,j 

j = l 

where aj denotes a vector of each second input pattern, and each second input feature panern \aJ is expressed 
55 as a product faj^ = Faj^ of F and in an equation. 

19. A pattern identity judging apparatus, comprising: 
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first pattern obtaining means ior obtaining a set of first teaching patterns of a plurality of teaching samples 
according lo a first pattern obtaining process; 

second pattern obtaining msans tor obtaining a set of second teaching patterns of the teaching samples ac- 
cording to a second pattern obtaining process differing from the first pattern obtaining process; 
feature extracting means for calculating a teaching pattern distribution from the set of first teaching patterns 
obtained by the first pattern obtaining means or the set of second teaching patterns obtained by the second 
panern obtaining means, calculating a teaching distribution of a perturbation between the set of first teaching 
panerns and the set of second leaching patterns, and calculating a feature extraction matrix, which minimizes 
an overlapping area between the teaching pattern distribution and the teaching penurbation distribution, from 
the teaching pattern distribution and the teaching penurbation distribution; 

feature pHllcrn calculnling means for receiving h lirsl inpul pattern of a first inpul sample according lo Ihc first 
pattern obtaining process, receiving a second input pattern of a second input sample acconding to the second 
pattern obtaining process, calculating a tirst input feature pattern o1 the first input sample from the first input 
pattern according to the feature extraction matrix calculated by the feature extracting means to nnake the first 
input feature pattern independent of the first pattern obtaining process and the second pattern obtaining proc- 
ess, and calculating a second input feature pattern of the second input sample from the second input pattern 
according to the feature extraction matrix to make the second input feature pattern independent of the first 
pattern obtaining process and the second pattern obtaining process; and 

identity judging means lor collating the first input feature pattern calculated by the feature pattern calculating 
means \vith the second input feature pattern calculated by the feature pattern calculating means to estimate 
a similarity between the first input sample and the second inpul sampIO: and judging that the first input sample 
is identical with the second input sample in cases where the similarity is high. 

20. A pattern identity judging apparatus according to claim 1 9 in which the leature extracting means comprises 

pattern covartance assuming means for calculatii'ig a teaching pattern covariance matrix of a pattern sample 
space from the set of first teaching patterns or the set of second teaching patterns and assuming the teaching 
pattern covariance matrix as the teaching pattern distribution; 

pallcrn pcrlurbalicn calculating means (or calculating a leaching pallcrn pcrturbalion bclwccn one first leach- 
ing pattern of one teaching sample and one second teaching pattern of the teaching sample for each teaching 
sample; 

perturbation covariance assuming means for assuming a teaching perturbation covariance matrix from the 
teaching pattern perturbations calculated by the pattern perturbation calculating means as the teaching per- 
turbation distribution; 

both-diagonalizing matrix calculating means for calculating a both-diagonalizing matrix, which diagonalizes 
both the teaching pattern covariance matrix assumed by the pattern covariance assuming means and the 
teaching perturbation covariance matrix assumed by the perturbation covariance assuming means, from the 
teaching pattern covartance matrix and the teaching perturbation covariance matrix; 

diagonal matrix producing means for diagonalizing the teaching pattern covariance matrix assumed by the 
pattern covariance assuming means according to the both-diagonalizing matrix calculated by the both-diag- 
onalizing matrix calculating means to produce a diagonal matrix of the teaching pattern covariance matrix, 
and diagonalizing the teaching perturbation covariance matrix assumed by the perturbation covariance as- 
suming means according to the both-diagonalizing matrix to produce a diagonal matrix of the teaching per- 
turbation covariance matrix; 

amplitude re-transformation matrix calculating means for calculating an amplitude re -transformation matrix, 
which again transtorms a referential pattern covariance matrix indicated by the set of referential feature pat- 
loms lo be calculated by Ihc rcfcrcnlial fcalurc pallcrn calculating means lo adjust amplitudes ol diagonal 
elements of the referential pattern covariance matrix after the referential pattern covariance matrix is trans- 
formed by the both-diagonalizing matrix calculated by the bolh-diagona'lizing matrix calculating means to be 
diagonalized, from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix calculated by the both-diagonalizing 
matrix calculating means and the amplitude re-transtormaticn matrix calculated by the amplitude re-trai^sfor- 
mation matrix calculating means. 

21. A panem identity judging apparatus, comprising: 

tirst pattern obtaining means for obtaining a set of tirst teaching patterns ol a plurality of teaching samples 
according to a first pattern obtaining process; 
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seconci pattern obtaining means for obtaining a group of second teaching patterns according to a second 
pattern obtaining process differing fronn the first pattern obtaining process for each teaching sample; 
teaturs extracting nneans tor calculating a teaching pattern distribution from the set of tirst leaching patterns 
obtained by the first pattern obtaining nneans or the groups of second leaching patterns obtained by the second 
pattern obtaining means^ calculating a teaching distribution of a perturbation between one first teaching pattern 
of one leaching sample and the group of second teaching pattertis of the teaching sample tor each teaching 
sample, calculating an average teaching perturbation distribution from the leaching perturbation distributions, 
and calculating a feature extraction matrix, which minimizes an overlapping area between the teaching pattern 
distribution and the average teaching perturbalion distribution, from the teaching pattern distribution and the 
average teaching perturbation distribution; 

foalurc prillcrn calculHling mc^ns for receiving h first input pHltcrn of rj first input sample rjccording to Ihc Hrsl 
pattern obtaining process^ receiving a second input pattern ci a second input sample according to the second 
pattern obtaining process, calculating a first input feature pattern of the tirsi input sample from the first input 
pattern according to the feature extraction nnatrix calculated by the feature extracting means to make the first 
input feature pattern independent of the first pattern obtaining process and the second pattern obtaining proc- 
ess, and calculating a second input feature pattern of the second input sample from the second input pattern 
according to the feature extraction matrix to make the second input feature pattern independent of the first 
pattern obtaining process and the second pattern obtaining process; and 

identity judging means tor collating the first input feature pattern calculated by the feature pattern calculating 
means ^vith the second input feature pattern calculated by the feature pattern calculating means to estimate 
a similarity between the first input sample and the second input sample, and judging that the first input sample 
is identical with the and judging that the first input sample is identical with the second input sample in cases 
where the similarity is high. 

22. A pattern identity judging apparatus according 1o claim 21 in which the feature extracting means comprises 

pattern covariance assuming means for calculating a leaching pattern covariance matnx of a pattern sample 
space from the sel of first teaching patterns or the groups of second teaching patterns and assuming the 
lc^^chi^g pHllorn covririancc mnlrix rjs Ihc IcHching pHllcrn distribution: 

pattern perturbation calculating means for calculating a group of teaching pattern perturbations between one 
tirst teaching pattern of one leaching sample and one group of second teaching pattems of the teaching sample 
tor each teaching sample; 

pertu*ation covariance assuming nneans for calculating a teaching perturbation covariance matrix from one 
group of teaching pattern perturbations calculated by the pattern perturbation calculating means for each 
teaching sample, calculating an average teaching perturbation covariance matrix from the teaching perturba- 
tion covariance matrices and assuming the average teaching perturbation covariance matrix as the average 
teaching perturbation distribution; 

both-diagonalizing matrix calculating means for calculating a both-diagonalizing matrix, which diagonalizes 
both the teaching pattern covariance matrix assumed by the pattern covariance assuming means and the 
average teaching perturbalion covariance matrix assumed by the perturbation covariance assuming means, 
from the teaching panem covariance matrix and the average leaching perturbation covariance matrix: 
diagonal matrix producing means for diagonalizing the teaching pattern covariance matrix assumed by the 
pattern covariance assuming means according to the both-diagonalizing matrix calculated by the both-diag- 
onalizing matrix calculating means to produce a diagonal matrix of ihe teaching pattern covariance matrix, 
and diagonalizing the average teaching perturbation covariance matrix assumed by the perturbation covari- 
ance assuming means according to the both-diagonalizing matrix to produce a diagonal matrix of the average 
ICHching pcrlurbfJlion covHriHncc mntrix; 

amplitude re-transformation matrix calculating means for calculating an amplitude re -transformation matrix, 
which again transforms a referential pattern covariance nnatrix indicated by the set of referential feature pat- 
terns to be calculated by the referential feature pattern calculating means to adjust amplitudes of diagonal 
elements of the referential pattern covariance matrix after the referential pattern covariance matrix is trans- 
formed by the both-diagonalizing matrix calculated by the both-diagonalizing matrix calculating means to be 
diagonalized, from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix calculated by ih a both-diagonalizing 
matrix calculating means and the amplitude re-transformaticn matrix calculated by the amplitude re-transfor- 
malion matrix calculating means. 

23. A recording medium for recording a software program of a pattern recognizing method executed in a computer, 
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the pattern recognizing nriethod, ccmprising the steps o1: 

obtaining a set of tirst teaching patterns of a plurality ol teacliing sannples according to a first pattern obtaining 
process; 

obtaining a set of second teaching pattenns of the teaching samples according to a second pattern obtaining 
process dilfering from the first pattern obtain i rig process: 

calculating a teaching patteoi distribution trom the set of first teaching patterns or the set of second teaching 
patterns; 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of 
second teaching patterns; 

crjiculaitng H fcaluro CKlrHclicn mninx, which minimi/cs an ovorlripping area bclwccn the ioHching pHllcrn 
distribution and the teaching perturbation distribution, Irom the teaching pattern distribution and the teaching 
perturbation distribution; 

obtaining a set of referential patterns of a plurality of reterentiaJ sannples according to the first pattern obtaining 
process; 

calculating a set of referential feature patterns of the referential sannples ironn the set of refer erUial patterns 
according to the feature extraction nnatrix, the set of referential feature patterns being independent of the first 
pattern obtaining process and the second pattern obtaining process; 

receiving an input pattern of an input sample according to the second pattern obtaining process; 
calculating an input feature pattern of the input sample from the input pattern according to the feature extraction 
matrix; 

selecting a specific referential feature pattern most similar to the input feature pattern from the set of referential 
feature patterns; and 

recognizing a specific referential sample corresporiding to the specific referential feature pattern as the input 
sample. 

24. A recording medium according to claim 23 in which the step of calculating a teaching pattern distribution comprises 
the steps of 

calculating a teaching pattern covariance matrix of a pattern sample space from the set of first teaching patterns 
or the set of second teaching patterns; and 

assuming the teaching pattern covariance matrix as the teaching pattern distribution, 

the step of calculating a teaching distribution of a perturbation comprises the steps of 

calculating a teaching pattern perturbation between one first leaching pattern of one teaching sample and one 

second teaching pattern of the teaching sample for each teaching sample; 

calculating a teaching perturbation covariance matrix from the teaching pattern perturbations of the teaching 
samples: and 

assuming the teaching perturbation covariance matrix as the teaching perturbation distribution, and 
the step of calculating a feature extraction matrix comprises the steps of 

calculating a bolh-diagcnalizing rriatrix. which diagonalizes both the teaching pattern covariance matrix and 
the teaching perturbation covariance matrix, from the teaching pattern covariance matrix and the leaching 
perturbation covariance matrix; 

diagonalizing the teaching pattern covariance matrix according to the both-diagonal tzing matrix to produce a 
diagcrial matrix of the teaching patteni covariance matrix: 

diagonalizing the teaching perturbation covariance matrix according to the both<iiagonalizing matrix to pro- 
duce a diagonal matrix of the teaching perturbation covariance matrix; 

c^ilculHltng Hn Hmpliludc ro-lrHnsformrilion malrin, which HQ»\n transforms h rcfcrcnlial pallcrn covHrinncc 
matrix indicated by the set of referential feature patterns to adjust amplitudes ot diagonal elements of the 
referential pattern covariance matrix after the referential pattern covariance matrix is transformed by the both- 
diagonallzing matrix to be diagonallzed, from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix and the annplitude re-transformation 
matrix. 

25. A recording medium according to claim 23 in which the step ol calculating a teaching distribution ol a perturbation 
comprises the step of 

calculating a teaching perturbation distribution between one first teaching pattern of one teaching sample and 
one second teaching pattern of the teaching sample for each teaching sample, 
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tha step of calculating a feature extraction matrix comprises the step oi 

calculating a feature extraction matrix, which minimizes an overlapping area between a teaching pattern dis- 
tribution ot one teaching sample and the teaching perturbation distribution o1 the teaching sample, from the 
teaching pattern distribution and the teaching perturbation distribution of the teaching sample lor each teaching 
sample, 

the step of obtaifiing a set o1 referential patterns comprises the step of 

obtaining a set of referential patterns of the teaching samples according to the first pattern obtaining process 
or the second pattern obtaining process. 

the step of calculating a set of referential feature patterns comprises the step of 

calculating one referential feature panern of one teaching sample from one referential panern of the teaching 
sample according Lo Ihc feature cxtrnclion matrix o( the leaching sample for each leaching SHmpIc, 
the step of calculating an input feature pattern comprises the step of 

calculating an input feature pattern of the input sample from the input pattern according to the feature extraction 

matrix of one teaching sannple for each teaching sample, and 

the step of selecting a specific referential feature pattern comprises the steps of 

estimating a similarity between one input feature pattern corresponding to one teaching sample and one ref- 
erential feature pattern of the same teaching sample; and 

selecting a specific referential feature pattern of a specific teaching sample most similar to the input feature 
pattern corresponding to the leaching sample from the set of referential feature patterns. 

26. A recording medium according to claim 23 in which the step of obtaining a set of second teaching patterns com- 
prises the step of 

obtaining a group of second teaching patterns for each teaching sample. 

the step of calculating a teaching distribution of a perturbation comprises the step ot 

calculating a teaching perturbation distribution behveen one first teaching pattern of one teaching sample and 
one group ot second teaching patterns of the teaching sample lor each teaching sample, 
the step of calculating a feature extraction matrix comprises the step of 

calculating a fcalurc cxlfHclion matrix, v^hich minimi^'cs an overlapping area bclvuccn a teaching pallcrn dis- 
tribution of one leaching sample and the teaching perturbation distribution o1 the leaching sample, from the 
teaching pattern distribution and the teaching perturbation distribution of the teaching sample for each teaching 
sample^ 

the step of obtaining a set of referential patterns comprises the step of 

obtaining a set of referential patterns of the teaching sample according to the first pattern obtaining process, 
the step of receiving an input pattern comprises the step of 

obtaining a group of input patterns of the input sample according to the second pattern obtaining process, 
the step of calculating an input feature pattern comprises the steps of 

calculating a group of input feature patterns of the input sample from the group of input patterns according to 
the feature extraction matrix of one teaching sample for each teaching sample; and 

calculating an average input feature pattern from the group of input feature patterns for each leaching sample, 
and the step of selecting a specific referential feature pattern comprises the steps of 

estimating a similarity between one average input feature pattern coi responding to one teaching sample and 
one average referential feature pattern of the same teaching sample; and 

selecting a specific average referential feature pattern of a specific teaching sample most similar to the average 
input feature pattern corresponding to the teaching sample from the set of average referential feature patterns. 

27. A recording medium for recording a software program of a pattern identity judging method executed in a compulcr, 
the pattern identity judging method, comprising the steps of: 

obtaining a set of first teaching patternsfrom a plurality of teaching samples accordingto a first panern obtaining 
process; 

obtaining a set of second teaching patterns from the teaching samples according to a second pattern obtaining 
process differing from the first pattern obtaining process: 

calculating a teaching pattern distribution from the set of first teaching patterns or the set of second teaching 
patterns; 

calculating a teaching distribution of a perturbation between the set of first teaching patterns and the set of 
second teaching patterns: 

calculating a feature extraction matri\', which minimizes an overlapping area between the teaching pattern 
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distribulion and the teaching perturbation distribution, Irom the teaching pattern distribution and the leaching 
perturbation distribution; 

receiving a tirst input pattsrn of a first input sample according to the first pattern ot3laining process; 
calculating a first input teature pattern of the first input sannple frcnn the first input pattern according to the 
teature extraction mautx, the first input feature pattern being independent oi the first pattern obtaining process 
and the second pattern cbtabitng process; 

receiving a second input pattern of a second input sannple according to the second pattern obtaining process; 
calculating a second inputieature pattern o1 the second input sample liom the second input pattern according 
to the feature extraction matrix, the second input feature pattern being independent of the first pattern obtaining 
process and the second pattern obtaining process; 

coilaling Ihc firsl inpul fcHluro pHtlcrn with Iho soccnd input foHlurc pallorn to cslimrtlc ^ simitarily bclwccn 
the first input sample and the second input sample; and 

judging that the tirst input sample is identical with the second input sannple in cases where the similarity is high. 

28. A recording medium according to claim 27, in ^vhich the step of calculating a teaching pattern distribution comprises 
the step of 

assuming a teaching pauern covariance matrix of a pattern sample space as the teaching pattern distribution, 
the step of calculating a teaching distribution o1 a perturbation comprises the steps o1 
calculating a teaching pattern perturbation between one first teaching pattern of one teaching sample and one 
second teaching pattern of the leaching sample tor each teaching sample; and 

assuming a teaching perturbation covariance n^trix from the teaching pattern perturbations as the teaching 
perturbation distribution, and 

the step of calculating a feature extraction matrix comprises the steps cf 

calculating a both-diagonalizing matrix, which diagonalizes both the teaching pattern covariance matrix and 
the teaching peiturbation covariance matrix, from the teaching pattern covariance matrix and the teaching 
perturbation covariance matrix; 

diagohalizing the teaching pattern covariance matrix according to the both-diagonaiizing matrix to produce a 
dingonHl mHlrix of Ihc IcHching pHllcrn covHrinnco mHlrix; 

diagonalizing ihe teaching perturbation covariance matrix according to the both-diagonalizing matrix to pro- 
duce a diagonal matrix of the teaching perturbation covariance matrix; 

calculating an amplitude re-transformation matrix, which again transforms a referential pattern covariance 
matrix indicated by the set of referential teature patterns to adjust annpiitudes of diagonal elements of the 
referential pattern covaiiaiKe matrix after the referential pattern covariance matrix is transformed by the both- 
diagonalizing matrix to be diagonalized, from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrix and the amplitude re-transformation 
matrix. 

29. A recording medium according to ctaim 27. in which the step of obtaining a set ot second teaching patterns com- 
prises the step ol 

obtaining a group of second teaching patterns for each leaching sample, 
the step of calculating a teaching pattern distribution comprises the steps of 

calculating a teaching pattern covariance matrix of a pattern sample space from the set ol first teaching patterns 
or the set of second teaching patterns: and 

assuming the teaching pattern covariance matrix as the leaching pattern distribution, 
Ihc step of cHlculHting h IcHching dislribulion of h pcrlurbnlion comprises the slcps of 

calculating a group cf teaching panern penurbaticns between one firsl teaching pattern cf one teaching sample 
and one group ot second leaching patterns of the teaching sample lor each teaching sample; 
calculating a teaching perturbation covariance matrix 1rom one group of teaching pattern perturtsalions ol one 
teaching sample tor each teaching sample; 

calculating an average teaching penurbation covariance matrix from the teaching perturbation covariance 
matrices; and 

assuming the average teaching penurbation covariance matrix as the teaching perturbation distribution, 
the step of calculating a teaching distribution of a penurbation comprises the steps of 
calculating a both-diagonalizing matrix, which diagonalizes both the teaching pattern covariance matrix and 
the teaching perturbation covariance matrix, from the leaching pattern covariance matrix and the teaching 
perturbation covariance matrix; 
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diacjonalizing the teaching patlem covartance matrix according to the both-diagonalizing matrix to produce a 
diagonal matrix of the teaching pattern covariance matrix: 

diagonalizing the teaching perturbation covariance matrix according to the both-diagonalizing matrix to pro- 
duce a diagonal matrix of the teaching perturbation covariance matrix: 

calculating an amplitude re-transformation matrix, which again transforms a referential pattern covariance 
matrix indicated by the set of reterential feature patterns to adjust amplitudes of diagonal elements of the 
referential pattern covariance matrix after the referential pattern covariance matrix Is transformed by the both- 
diagonalizing matrix to be diagonalized. from the diagonal matrices; and 

calculating the feature extraction matrix from the both-diagonalizing matrixand the amplitude re-transformation 
matrix. 
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